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EXECUTIVE  SUMMARY 


PROBLEM 

In  recent  years,  virtual  reality  (VR),  often  called  virtual  environments  (VE),  has  received  an 
enormous  amount  of  attention  among  traioing  developers.  This  is  due  in  part  to  media  hype  over 
applications  of  VR  in  the  entertainment  industry,  but  even  more  so  because  training  developers 
recognize  the  potential  of  VR  as  a  flexible  and  effective  training  medium.  Training  effectiveness 
evaluations  of  VR  training  systems  can  he^  define  the  appropriate  use  of  VR  technologies  to 
achieve  this  potential.  Moreover,  as  with  all  training  media,  the  potential  of  VR  can  not  be 
achieved  imless  VR  training  systems  are  usable  by  the  instructors  and  trainees  for  whom  they  are 
developed. 

A  prime  candidate  task  area  for  examining  the  effectivness  and  usability  of  VR  systems  is  the 
training  of  submarine  surfeced  ship  handling.  Although  land-based  shnulator  fecflides  currently  exist 
for  training  Submarine  Piloting  and  Navigation  teams,  these  ^sterns  do  not  provide  detailed  harbor  and 
channel  ship  handling  training  for  the  Officer  of  the  Deck  (OOD).  OOD  training  is  primarily  obtained 
from  on-the-job  e?q)erience,  which  is  adversely  inpacted  by  the  operational  constraints  of  the 
Submarine  Force,  and  the  limited  sur&ced  steaming  time  of  submarines.  Training  that  will  expose 
junior  officers  to  a  variety  of  geographical  and  environmental  conditions  is  very  limited  since  most 
Commanding  Officers  place  their  most  experienced  OODs  on  watch  during  these  challenging 
evolutions.  Therefore,  an  akemative,  simulation-based  training  capability  is  needed.  A  VR-based 
gmulation  may  provide  this  necessary  capability  if  it  is  both  effective  and  user-fiiendly. 

OBJECTIVE 

The  objective  of  the  Virtual  Environment  for  Submarine  Shp  Handling  and  Piloting  Training 
(VESUB)  project  is  threefold:  ( 1)  to  develop,  demonstrate,  and  evaluate  the  training  potential  of  a 
stand-alone  virtual  reality-based  ^stem  for  OOD  training;  (2)  to  determine  if  this  system  can  be 
integrated  with  existing  Submarine  Piloting  and  Navigation  (SPAN)  training  Emulators,  and  (3)  to 
determine  the  viability  of  virtual  reality  technology  as  a  training  tool  This  r^ort  looks  at  the  user- 
oriented  deagn  issues  for  the  VESUB  Instractor/Operator  Station  (lOS).  The  ability  of  Navy 
instmctors  to  use  the  VESUB  tystem  will  be  critical  to  the  effectiveness  of  the  training  system.  The 
results  of  the  user-oriented  design  anafysis,  as  documented  in  this  report,  will  be  used  in  conjunction 
with  the  results  of  training  effectiveness  evaluations  (to  be  documented  in  a  separate  publication)  so  the 
deagn  of  the  follow-on  operational  VESUB  system  can  incorporate  the  latest  technologies  and  also  to 
be  as  user-fiiendly  as  posable. 

APPROACH 

The  students  from  a  graduate  level  “Human-Computer  Interaction  Design  and  Evaluation” 
class  at  the  University  of  Central  Florida  conducted  usability  analyses  of  the  VESUB 
Instractor/Operator  Station  (lOS)  during  the  formative  evaluation  phase  of  system  development. 
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Ttese  analysts  applied  Nielsen’s  (1993)  design  criteria  to  identify  niimerous  lOS  usability 
violations.  The  results  of  these  analyses  were  fiirther  analyzed  by  the  VESUB  research  team  to 
determine  the  most  critical  usability  design  violations  that  require  correction  ia  the  operational 
VESUB  system 

RESULTS 

Four  separate  analysis  reports  were  produced  for  class  credit  and  provided  to  the  VESUB 
research  team  The  reports  documented  both  major  and  minor  usability  design  violations.  Some 
of  these  violations  were  corrected  in  later  software  iterations.  A  post-hoc  analysis  of  the  high 
priority  design  violations  that  could  not  be  corrected  in  the  demonstration  system  identified 
twenty-two  areas  where  the  design  of  the  operational  VESUB  system  lOS  could  be  irtq)roved. 

RECOMMENDATIONS 

Of  the  twenty-two  lOS  design  violations  that  are  discussed,  one  has  been  corrected  in  the 
demonstration  system,  five  have  been  partially  corrected,  and  sixteen  require  correction  in  the 
operational  VESUB  system  The  training  mq)lications  of  these  design  violations  are  discussed. 

In  addition  to  the  recommended  mq)rovements  to  the  design  of  the  operational  VESUB 
systems,  five  general  areas  for  research  are  recommended  to  inq)rove  the  usability  of  any  fixture 
instructor  station.  These  areas  were  chosen  on  the  basis  of  the  lessons  learned  during  VESUB 
development  and  their  potential  to  improve  the  performance  of  the  instructor  at  the  lOS.  They 
are;  1)  standardization  of  graphical  user  interfaces;  2)  development  of  on-line  help  for  the 
instructor;  3)  development  of  embedded  instructor  training;  4)  application  of  task  analysis 
methods  to  instructor  tasks;  and  5)  research  on  the  use  of  voice  recognition  at  the  lOS. 
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INTRODUCTION 


PROBLEM 

In  recent  years,  virtual  reality  (VR),  sometimes  called  virtual  environments  (VE),  has  received 
an  enormous  amomit  of  attention  among  training  developers.  This  is  due  in  part  to  media  hype 
over  applications  of  VR  in  the  entertainment  industry,  but  even  more  so  because  training 
developers  recognize  the  potential  of  VR  as  a  flexible  and  effective  training  medium  Training 
effectiveness  evaluations  of  VR  training  systems  can  help  define  the  appropriate  use  of  VR 
technologies  to  achieve  this  potential.  Moreover,  as  wMi  all  training  media,  the  potential  of  VR 
can  not  be  achieved  unless  VR  training  systems  are  usable  by  the  instructors  and  trainees  for 
whom  they  are  developed.  The  Virtual  Environment  for  Submarine  OOD  Ship  Handling  Training 
(VESUB)  technology  demonstration  system  provides  an  opportunity  to  evaluate 
Instructor/Operator  Station  (lOS)  design  alternatives  for  both  the  operational  (follow-on) 

VESUB  system  and  other  future  VR-based  training  systems. 

OBJECTIVE 

This  is  the  first  in  a  series  of  reports  to  document  the  results  of  the  VESUB  project.  It 
discusses  the  user-oriented  design  issues  in  the  development  of  the  instructor  station  for  a  VR 
training  system  technology  demonstration  for  submarine  ship  handling,  called  VESUB  .  Althou^ 
the  design  issues  are  discussed  in  the  context  of  the  VESUB  technology  demonstration  system, 
they  are  applicable  to  any  computer-based  training  system  that  includes  an  instructor  station.  The 
results  of  this  usability  analysis  will  be  used  in  the  development  of  the  specifications  for  the 
operational  VESUB  system  This  analysis  should  also  serve  to  help  the  design  of  the  instractor 
stations  for  other  future  training  systems.  Other  reports  will  document  the  results  of  the  VESUB 
training  effectiveness  evaluations  and  lessons  learned  fi^om  the  project.  It  is  hoped  that  these 
reports  will  provide  guidance  to  increase  the  training  effectiveness  of  future  VR  training  systems. 

ORGANIZATION  OF  THE  REPORT 

The  first  sections  provide  a  brief  overview  of  the  submarine  ship  handling  task  and  the  need  for 
the  VESUB  training  system  A  brief  overview  of  VR  training  research  explains  why  submarine 
ship  handling  was  chosen  as  the  technology  demonstration  task.  A  description  of  the 
developmental  phases  and  formative  evaluations  of  the  VESUB  technology  demonstration  system 
and  the  need  for  usability  analyses  are  also  provided.  The  next  section  is  a  discussion  of  user- 
oriented  design  approaches  and  the  design  heuristics  that  were  applied  to  the  VESUB  Instructor 
Station.  The  usability  analysis  method,  results,  and  discussion  are  presented  in  the  subsequent 
sections.  Finally,  recommendations  for  corrections  of  VESUB  design  violations  and  areas  for 
fixture  research  to  improve  the  user-oriented  design  of  instructor  stations  are  provided. 
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THE  VIRTUAL  ENVmONMENT  FOR  SUBMARINE  SHIP  HANDLING  AND 
PILOTING  TRAINING  TECHNOLOGY  DEMONSTRATION  SYSTEM  (VESUB) 

BACKGROUND 

Land-based  simulator  fecilities  currently  exist  for  training  Submarine  Piloting  and  Navigation  teams. 
Tbese  systems  do  not,  however,  provide  detailed  harbor  and  channel  ship  handling  training  for  the 
Officer  of  the  Deck  (OOD).  OOD  training  is  primarily  obtained  from  on-the-job  experience,  ^^hich  is 
adversely  iirpacted  by  the  operational  constraints  of  the  Submarine  Force,  and  the  limited  sui&ced 
steaming  time  of  submarines.  Training  of  junior  officers  under  a  variety  of  geographical  and 
environmental  conditions  is  very  limited  since  most  Commanding  Officers  place  their  most  experienced 
OOD  on  watch  during  these  challenging  evolutions.  Therefore,  an  alternative,  simulation-based 
training  capability  is  needed  and  may  be  met  effectively  and  economically  using  virtual  reality  (VR). 

Research  in  Virtual  Reality 

VR  has  received  considerable  publicity  in  recent  years  from  the  entertainment  industry  (e.g..  Brill, 
1994;  Gradecki,  1994)  and  the  popular  press  (e.g.,  Rinegold,  1992;  Kalansky,  1993;  Burdea  &  Coiffet, 
1994).  VRhas  also  generated  conaderable  interest  in  the  military  (e.g.,  McVey,  1993;  Cook,  1994; 
Larrpton,  Knerr,  Goldberg,  Bliss,  Moshell,  &  Blau,  1994;  Knerr,  Goldberg,  Lampton,  Witmer, 
Bliss,  Moshell,  &  Blau,  1994)  and  NASA  (e.g..  Null  &  Jaokins,  1993). 

Traditional  military  training  systems  have  used  operatiorral  equipment,  large  training  ranges,  or 
expensive  filiation  equpment.  VR  affords  the  potential  to  greatly  reduce  the  cost  of  training  systems 
because  it  can  provide  a  trainee  with  an  interfece  to  training  equpment  without  the  necessity  of 
replicating  expense  hardware.  Furthermore,  VR  traming  can  be  provided  on  a  range  of  tasks  using 
the  same  basic  trainee  interfece  by  changing  the  VR  software.  This  can  potentially  save  huge  amounts 
of  resources  'wiiich  would  otherwise  he  required  to  build  a  complete  new  phyacally  different  trainee 
interfece.  VR  also  offers  the  potential  to  provide  new  and  innovative  training  strategies  that  can  only 
be  applied  in  a  virtual  world.  For  excanple,  a  trainee  could  view  the  inade  of  a  fuel  ^stem  from  the 
perspective  of  a  molecule  of  gasoline;  he  or  she  could  grow  to  the  aze  of  a  giant  to  view  a  tactical 
engagement  from  above,  then  junp  from  one  shp  to  another  for  various  points  of  view;  or  a  trainee 
could  engage  in  team  training  with  virtual  team  members.  However,  the  potential  of  VR  training  can 
not  be  achieved  unless  researchers  demonstrate  the  effectiveness  of  VR  training  syaems. 

Research  in  VR  is  being  conducted  at  various  laboratories  (e.g.,  the  Army  Research  Institute,  and 
the  Navy  Post  Graduate  School)  in  three  modality  areas:  visual,  auditory,  and  haptic.  Visual  research 
includes  efforts  to  improve  display  devices  (e.g.,  head-mounted  displays  [HMDs]  and  “cave” 
projection  screens)  and  head  tracking  systems.  The  visual  modality  is  the  most  highfy  advanced  area  of 
VR  research.  Among  other  questions,  auditory  research  is  excamining  the  effectiveness  of  the  use  of 
highly  realistic  and  3-D  localized  sounds  in  virtual  worlds.  Research  on  haptic  displays  seeks  to 
inprove  the  simulation  of  both  the  tactile  (cutaneous)  and  kinesthetic  (muscle  and  joint)  aspects  of 
virtual  touch.  It  is  the  least  mature  of  the  three  VR  modalities. 
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TheVESIJBPrn^ftnt 

The  Virtual  Environment  for  Submarine  Sh^  Handling  and  Piloting  Training  (VESUB)  is  a  Navy 
Manpower,  Personnel,  and  Training  R&D  project  with  the  goal  to  develop,  demonstrate,  and  evaluate 
the  trammg  effectiveness  of  a  VR-based  system  for  sur&ced  submarine  OOD  training  The  submarine 
^  handling  task  was  chosen  as  the  task  area  for  the  technology  demonstration  for  three  reasons;  1) 
mere  is  a  clear  Navy  need  for  this  training;  2)  submarine  handling  is  a  visually  dependent  task-  and 

3)  the  visual  modality  IS  the  most  mature  ofthe  three  areas  of  VR  development.  If  effective  VESUB 

has  the  potential  to  reduce  the  number  of  sh^  handing  mishaps,  save  lives,  and  reduce  property  loss. 
Based  on  the  results  ofthe  VESUB  R&D  effort,  the  Navy  will  procure  five  operational  VESUB 
systems  to  be  installed  at  the  main  submarine  trainkg  fedlities  throughout  the  United  States. 

The  VESUB  system  was  developed  by  Nichols  Advanced  Marine  (formerly  Advanced  Marine 
Enterprises,  [AME])  under  contract  and  direction  of  a  research  team  at  the  Naval  Air  War&re  Center 
Trar^gSysteriKDhdsion(NAWCTSD)in(>lando,FL.  The  research  team  includes:  research 
Phenologists,  visual  engineers,  con5)uter  engineers,  and  submarine  subject  matter  experts  To  save 
toe  and  money,  VESUB  was  developed  by  leveraging  AME’s  Virtual  Ship,  a  commerdal  product 
that  has  been  used  for  several  years  to  train  surfece  shp  handling  tasks,  histallations  of  Virtual  Shro 
use  large  projection  screens  to  display  the  visual  scene  and  firll  bridge  mock-ups  for  the  trainee 

mterfece.  These  systems  also  require  several  full-time  instmetors  to  operate  their  very  complex 
mstractor  stations.  ^ 

Under  the  VESUB  project,  AME  has  modified  Virtual  Ship  by  using  head-mounted  diqrlay  (HMD) 
md  mag^c  head  tracking  technologies  to  present  the  visual  scene  to  trainees  (see  ^stem  deserrotion 
below)^  The  VESUB  project  has  also  required  AME  to  enhance  the  level  of  detail  and  fidelity  in  the 
visu^  databases  fax  beyond  previous  ^sterns.  Furthermore,  in  addition  to  advancements  in  the 
hardware  and  software  technologies  used  in  Virtual  Shp,  VESUB  has  introduced  numerous 
mprovements  in  the  design  ofthe  instmetor  inter&ce  so  it  can  be  ea^  used  by  Navy  instmetors 
However,  schedule  and  budget  constraints  have  limited  the  scope  of  these  improvements  in  the 
technology  demonstration  VESUB  system,  but  the  results  of  tt^  anafysis  can  be  used  to  ensure  that 
the  operational  VESUB  system  is  designed  to  be  as  user-fiiendfy  as  possible. 

VESUB  SYSTEM  OVERVIEW 

The  VESUB  technolo^  demonstration  system  uses  a  high-resohrtion  head  mounted  display 
(Hlvp)  to  provide  to  trainee  with  a  simulated  360  degree  visual  environment  containing  all  ofthe 
requued  cues  associated  vto  harbor  and  channel  navigation  as  well  as  accurate  cultural  features  and 
varying  ravironmental  conditions.  Voice  recognition  and  synthesis  are  used  to  provide 
communications  training.  Visual  scene  rendering,  computation  of  harbor  currents  and  wind  effects 
and  operation  of  own  shp  and  other  traffic  dips  require  state-of-the-art  computer  ^sterns  and  ’ 

software  packages.  Appendix  A  lists  and  describes  to  hardware  and  software  used  in  VESUB. 
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Figure  1  shows  an  artist’s  depiction  of  the  VESUB  system  On  the  right  ade  of  the  figure,  an 
instructor  is  shown  seated  in  front  of  the  three  monitors  at  the  histnictor/Operator  Station  (lOS). 
Screens  on  two  of  the  lOS  monitors  are  used  to  modify  and  control  training  scenarios.  The  third 
monitor  is  used  to  observe  the  virtual  world  that  the  trainee  sees  throu^  the  HMD  during  a  training 
exercise.  This  view  allows  the  mstructor  to  evaluate  the  trainee’s  performance  in  real-time  and  provide 


Figure  1:  Artist’s  Representation  of  VESUB 

guidance  to  direct  the  trainee  to  look  for  the  correct  objects  to  support  the  task.  This  same  monitor  is 
used  before  an  exercise  to  create  the  framing  scenarios,  hi  Figure  1,  the  trainee  is  shown  standing  in 
the  bridge  mock-up,  wearing  the  HMD  and  communicating  with  a  ^ulated  navigation  team  via  a 
hand-held  microphone.  The  inset  depicts  a  typical  harbor  scene  as  the  trainee  views  it  through  the 
HMD.  The  visual  scene  di^lays  distant  objects,  including:  harbor  geogrq)hic  features,  navigation  aids, 
landmarks,  and  trafiSc  di^s.  It  also  displays  objects  near  the  trainee,  including:  a  representation  of  the 
bridge  area  (for  either  the  6881  or  the  726  classes  of  submarines),  and  much  of  the  equ^ment  needed 
to  he^  the  OOD  perform  the  diip  handling  task.  The  requirement  to  display  near  and  distant  objects 
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necessitated  the  use  of  new  visual  scene  management  processes  (e.g.,  large  area  database  management 
^stem)  that  were  not  needed  in  previous  sh^  handling  training  systems.  The  trainee  is  also  able  to  see 
sinplified  charts  and  other  piloting  information  on  request  via  the  voice  interfece.  When  the  trainee 
turns  his  head,  a  magnetic  head  tracker,  mounted  above  the  mock-up,  senses  the  movement  and  the 
con^uter  changes  the  visual  scene  ^propriately.  Thus,  for  example,  tbe  trainee  is  able  to  turn  to  the 
stem  and  observe  the  radder  move  in  response  to  an  issued  helm  order.  Visual  obstacles,  such  as 
lookouts,  periscopes,  and  radar  masts  can  be  displayed  at  the  discretion  of  the  instructor  to  make  the 
training  tads  more  realistic  by  forcing  the  trainee  to  look  around  them  to  see  critical  visual  cues. 

The  Submarine  Ship  Handling  Task 

A  submarine,  when  surfeced,  must  perform  according  to  the  ship  handling  procedures 
followed  by  surface  ships.  Current  submarine  OOD  drip  handling  training,  since  it  is  primarily 
conducted  on-the-job,  requires  each  ship’s  Commanding  Officer  (CO)  to  evaluate  the 
performance  of  junior  officers  (JOs)  prior  to  certifying  them  as  qualified  OODs.  In  the  early 
phases  of  the  VESUB  project,  submarine  COs  were  interviewed  to  determine  how  they 
accomplished  this  task,  hi  almost  every  case,  the  COs  responded  that  their  personnel  were 
qualified  when  they  had  developed  the  “Seaman’s  Eye.”  This  concept  was,  therefore,  used  as  the 
keystone  for  the  development  of  VESUB  training  objectives.  For  the  purposes  of  the  VESUB 
effort,  the  following  definition  of  “Seaman’s  Eye”  was  developed  to  guide  the  research. 


Seaman  s  Eve:  The  total  situation  awareness  of  the  ship  handling 
environment  and  the  ability  to  safely  maneitver  the  vessel  in  all  conditions. 

This  definition  was  still  too  general  to  serve  as  a  guide  for  the  development  of  VESUB  hardware 
and  software  requirements  and  training  objectives.  Therefore,  iterative  probing  of  subject  matter 
experts  (SMEs),  using  focused  group  discussions  at  Navy  training  facilities  and  the  NAWCTSD 
laboratory,  helped  reveal  a  variety  of  perceptual  and  cognitive  conqionents  that  make  up  this 
complex  concept.  Additional  group  discussions  with  the  SMEs  helped  the  VESUB  research  team 
to  organize  the  components  as  shown  in  Table  1. 

The  perceptual  conqionents  of  Seaman’s  Eye”  were  initially  used  to  determine  the  hardware 
requirements  for  the  system.  For  example,  the  requirement  to  locate  and  identify  navigation  aids 
(PI)  and  to  judge  distance  (P2)  required  that  the  trainee  be  able  to  see  the  objects  at  large 
distances  and  necessitated  the  use  of  a  high  resolution  HMD.  Tlie  requirement  to  visually  inspect 
the  rudder  necessitated  the  use  of  head  tracking  and  rapid  refresh  rate  of  the  visual  scene.  Details 
on  the  hardware  determination  for  VESUB  can  be  found  in  Hays,  Castillo,  Bradley,  and  Seamon 
(1997).  Both  the  perceptual  and  the  cognitive  conq)onents  were  used  to  develop  and  organize  the 
training  objectives  for  the  VESUB  system.  The  training  objectives,  shown  in  Appendix  A,  were 
used  to  develop  the  training  and  evaluation  criteria  for  the  VESUB  Training  Effectiveness 
Evaluations  (TEEs).  The  results  of  the  TEEs  will  be  documented  in  a  separate  report. 
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Table  1 

Components  of  “Seaman’s  Eye” 


PERCEPTUAL  COMPONENTS 

IP.  Locating  and  Identifying  Navigation  Aids 

2P.  Judging  Distance 

3P.  Identifymg  the  Start  and  Completion  of  Turns 

4P.  Locating,  Identifydng,  and  Avoiding  Obstacles 

5P.  Sense  of  Ship  ’  s  Responsiveness 

1  8P.  Detecting  and  Filtering  Communications  I 

COGNITIVE  COMPONENTS 

1C.  Understanding  the  Relationship  of  Visual  Cues  to  their  Representations  on  Charts 

2C.  Understanding  Relative  Size  and  Height/Range  Relationships,  and 

Angle  on  the  Bow  (AOB) 

3C.  Understanding  Advance  and  Transfer 

4C.  Understanding  the  Effects  of  Tides,  Currents,  Wind,  and  Seas 

5C.  Understanding  Rules  of  the  Road 

6C.  Understanding  Relative  Motion  (Direction  and  Speed) 

7C.  Understanding  Methods  to  Differentiate  and  Prioritize  Traffic  Contacts 

8C.  Understanding  Ship’s  Operation  Under  Harbor  Directives 

9C.  Understanding  Methods  to  Deal  with  Uncooperative  Traffic 

IOC.  Understandmg  Correct  Operation  of  Ship’s  Systems 

lie.  Understanding  When  and  How  to  Take  Corrective  Actions 

12C.  Understanding  Effective  Communication  Procedures 

VESUB  DEVELOPMENT  PHASES 

The  VESUB  project  proceeded  in  three  phases:  requirements  determination,  formative 
evaluations,  and  training  effectiveness  evaluations.  A  major  tool  used  during  the  requirements 
determination  phase  was  a  feasibility  demonstration  system  developed  under  the  NAWCTSD 
exploratory  research  Virtual  Environment  Training  Technology  project.  The  feasibility 
demonstration  system  included  a  simplified  harbor  scene  and  submarine  model,  which  was  viewed 
through  a  low  resolution  HMD.  This  system  afforded  Navy  subject  matter  experts  (SMEs)  a 
chance  to  experience  a  virtual  world.  After  the  SMEs  tried  the  feasibility  demonstration  system. 
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queai^aires  md  interviews  were  used  to  soUcit  detaUed  functional  requirements  for  the 
yhbUB  technology  demonstration  system  from  the  SMEs.  These  requirements  were 

^  ^  NAWCTSD  special  report  (Tenney,  Briscoe,  Pew,  Bradley,  Seamon,  &Hays 


The  formative  evaluation  phase  was  conducted  in  the  laboratory  at  NAWCTSD.  Whenever  AME 
produced  m  in^roved  iteration  of  the  VESUB  software  on  their  development  system,  it  was  installed 

n  «  m  laboratory  and  evaluated  against  the  functional  requirements 

The  VESUB  res^ch  team  included  retired  Navy  submarine  SMEs  \^ho  assisted  with  the  formative 
eva^tions  on  a  ^  basis.  Data  for  these  formative  evaluations  were  supplemented  by  inputs  from 
fleet  md  sc^ol  SMEs  who  were  brought  into  the  laboratory  when  their  operational  schedules 
permitted.  The  re^s  of  the  formative  evaluations  were  provided  to  the  software  developers  for  the 
next  astern  iteration.  The  formative  evaluations  focused  on  both  the  functionality  of  the  trainee 
mter&ce  (e.g.,  the  fidelity  of  objects  in  the  visual  scene  or  the  fimctionality  of  the  voice  recognition 

system),  and  the  usability  of  the  lOS.  The  results  of  the  usability  anafyses  are  presented  in  lato 
sections. 

The  tr^g  effet^eness  evaluations  (TEEs)  ofthe  VESUB  technology  demonstration  system  wffl 
be  conduct^  at  the  Submanne  Training  Facility  in  Norfolk,  Virginia  and  the  Navy  Submarine  School 
m  &oto^  Connecticut  duimg  the  Winter  and  Spring  of  1998.  The  TEEs  will  use  actual  Navy  trainees 
vanous  levels  of  experience  (novice  to  expert)  to  determine  the  ejffectiveness  ofthe  VESUB 

syst^rnd  almohe^  deterinine  how  the  technology  can  be  integrated  into  Navy  training.  The 

results  of  the  TEEs  and  other  lessons  learned  during  the  VESUB  project  will  be  documented  in 
separate  pubhcations. 


VESUB  AS  A  TRAINmC  SYSTEM 

A  trainmg  system  consists  ofthe  planned  interaction  of  people,  materials,  and  techniques  with 
Toof?  T  performance  as  measured  by  established  criteria  on  the  job  (Hays 

1992).  It  has  been  demonstrated  that  training  simulators  and  devices  are  more  effective  if  they  are 
mtegrate  mto  a  program  of  instruction  that  takes  advantages  of  their  capabilities  (Caro  1972- 
aro,  Isley,  &  Jolley,  1973).  It  has  also  been  demonstrated  that  the  effectiveness  of  simulators  is 
havered  by  the  lack  of  mstructor  training  on  the  use  of  the  simulator’s  training  features 
(Biersner,  1976).  An  effective  training  system  must  integrate  numerous  subsystems,  including; 
tranung  equqiment,  school  fecilities,  administration,  and  instructors  (see  Hays  1992  for  a 
discussion  of  training  systems  concepts).  A  lack  of  training  system  integratioi  has  been  shown  to 
co^romse  the  potential  of  training  devices  (Caro,  1977;  Hiitz  &  Puiifoy,  1980-  Iffland  & 
Whiteside,  1977).  ’ 


Although  VESUB  IS  a  complex  configuration  of  hardware,  software,  and  courseware  these 
constituents,  no  matter  how  well  designed,  are  not  sufficient  to  ensure  that  it  will  be  an  effective 
trainmg  system.  For  optimum  effectiveness,  VESUB,  or  any  other  training  system,  must  be 
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usable  by  the  instructors  who  will  operate  it.  Therefore,  the  lOS  design  must  be  determined  by 
the  needs  of  the  instructor,  as  a  critical  component  of  the  training  effectiveness  of  the  system 
The  following  sections  discuss  methods  of  usability  analysis,  and  describe  the  development  and 
formative  evaluation  of  the  VESUB  lOS. 

TRAINING  SYSTEM  USABBLITY 

The  effectiveness  of  any  training  system  can  be  coicpromised  by  the  lack  of  user  acceptance. 
Nielsen  (1993)  discusses  the  attributes  of  any  conoputer  system  that  contribute  to  its  acceptability. 
Figure  2  (adapted  from  Nielsen,  1993,  p.  25)  shows  one  model  of  these  attributes.  System 
acceptability,  shown  on  the  left  side  of  the  figure  is  first  broken  down  into  two  constituents: 
social  and  practical  acceptability.  Social  acceptability  involves  the  purpose  of  a  system  and  the 
context  in  which  it  is  used.  For  example,  a  training  system  could  be  designed  to  mq)rove  a 
criminal’s  shoplifting  performance.  However,  such  a  system  would  not  be  socially  acceptable  in 
conventional  social  settings. 


Social 

Acceptability 


F^ure  2:  Attributes  of  System  Acceptability 
(modified  from  Nielsen,  1993,  p.  25) 


The  other  attribute  of  acceptability  is  whether  the  system  is  practical.  This  involves  questions 
of  system  cost  and  system  reliability.  The  hardware  of  current  VR  systems,  including  VESUB,  is 
very  costly  and  somewhat  unreliable.  Among  the  goals  of  VESUB  and  other  VR  research 
programs  is  to  improve  system  reliability  and  reduce  costs.  Demonstrations  of  the  training 
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effectiveness  of  systems  like  VESUB  will  be  a  strong  stimulus  for  the  user  community  to  push  the 
con^uter  industry  toward  mj5)rovements  in  these  areas. 

Another  attribute  of  system  practicality  is  its  compatibility  with  other  existing  systems.  One  of 
the  goals  of  the  VESUB  project  is  to  attenipt  to  interface  its  modem  hardware  and  software  with 
the  older  hardware  and  software  of  the  Submarine  Hloting  and  Navigation  (SPAN)  trainer.  This 
is  a  major  challenge  because  each  system  is  based  on  umque  hardware  and  software 
configurations.  Although  not  the  focus  of  this  paper,  hardware  interfaces  and  data  compatihility 
we  areas  that  will  require  considerable  research  if  VR  systems  are  to  reach  their  potential  and  be 
integrated  into  fiiture  training  systems  and  the  training  community 

The  component  of  practical  acceptability  that  is  the  focus  of  this  paper  is  the  usefulness  of  the 
system.  Usefulness  addresses  “the  issue  of  whether  the  system  can  be  used  to  achieve  some 
desired  goal”  (Nielsen,  1993,  p.  24).  Usefulness  questions  may  address  i^stem  utihty  or  system 
usability  (Gradin,  1992).  Utility  q^uestions  ask  whether  tbe  system  functions  as  it  needs  to 
function.  For  exanqtle,  does  a  training  system  provide  fiie  necessary  functionality  to  meet  all 
major  traming  objectives.  This  is  a  major  issue  with  complex  conqruter-based  training  systems. 
Often  resources  are  spent  on  areas  of  system  functionality  that  make  the  system  perform  well  at 
ftade  shows  or  other  demonstrations,  but  resources  run  out  before  the  instmctional  functionality 
is  included  in  the  system  to  ensure  efiective  training.  Even  if  instructional  features  are  included  in 
the  ^stem,  it  still  may  not  be  useful  unless  the  user  is  able  to  efficiently  operate  it.  This  is  the 
usability  coirponent  of  usefiilness. 

Usability  questions  address  how  well  system  users  can  enqjloy  its  functionality.  For  training 
systems,  the  two  major  classes  of  users  are  the  trainees  and  the  instructors.  This  report  focuses 
on  the  usability  issues  confi-onted  by  the  instructor.  These  issues  are  especially  critical  for  Navy 
instructors,  who  do  not  receive  extensive  training  m  instructional  techniques.  If  a  training  system 
is  not  designed  to  be  easy  fijr  Navy  instructors  to  use,  it  is  likely  to  be  incorrectly  utilized,  under 
utilized,  or  perhaps  not  used  at  all. 

The  conq)onents  of  system  usability  are  diown  in  the  lower  right  section  of  Figure  2.  A  usable 
traming  system  should  be  easy  to  learn.  The  mstructor  should  not  have  to  spend  an  inordinate 
amount  of  time  learning  special,  system-unique  languages  or  procedures.  The  &ster  instructors 
can  learn  to  use  a  training  system,  the  more  rapidly  they  can  get  on  with  their  real  work, 
conducting  traming.  Once  the  instructor  has  learned  to  use  the  system,  it  ^ould  be  efficient  to 
use.  If  the  training  system  is  cumbersome,  the  instructor  will  qiend  inordinate  amounts  of  time 
and  energy  “drivmg”  the  system  and  not  be  able  to  concentrate  on  monitoring  the  trainee.  The 
operation  of  the  traming  system  should  also  be  easy  to  remember.  The  instructor  should  not  have 
to  relearn  the  system  each  time  he  or  she  accesses  a  system  function  after  some  period  of  time. 

The  training  system  should  also  be  designed  to  minimize  the  instructors  ’  errors  during  system 
use.  If  errors  do  occur,  it  should  be  easy  to  recover  from  them  Finally,  the  mstructor  should  find 
the  traming  s^^eai pleasant  to  use.  No  mstructor  will  continue  to  use  a  training  system  that  he  or 
she  finds  subjectively  impleasant. 
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No  training  system  is  automatically  usable.  This  is  eq)ecia]ly  true  of  coirq)lex  cort5)uter-based 
systems  like  virtual  environments.  The  goals  of  the  VESUB  project  included  demonstrations  of 
hardware  and  software.  However,  the  usability  of  the  VESUB  technology  demonstration  was 
paramount  fi'om  the  beginning  of  the  project.  Therefore,  the  formative  evaluations  of  the  VESUB 
system  included  Human  Computer  Interface  (HCI)  usability  analyses  of  the  lOS. 
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USABILITY  OF  THE  VESUB  INSTRUCTOR/OPERATOR  STATION  (lOS) 

As  new  technology,  VESUB  is  pushing  the  state-of-the-art  on  several  fronts.  The  level  of 
graphics  detail  and  need  for  rapid  screen  refresh  rate  as  the  trainee  moves  through  the  360  degree 
environment  places  heavy  con:q)utational  demands  on  the  system  These  computational  demands 
are  exacerbated  when  the  submarine  hydrodynamic  models  are  modified  by  the  real-time  effects  of 
cmrents,  wind,  sea  states,  and  other  factors.  Still  additional  coiiq)utational  demands  are  placed  on 
the  system  when  the  hydrological  properties  of  the  visual  scene  are  changed  (e.g.,  the  presence  of 
white  caps  with  increasing  wind).  The  efforts  of  talented  corcputer  programmers  have  been 
directed  towards  optimizing  VESUB ’s  hardware  resources  with  the  latest  programming 
techniques.  However,  it  is  not  computer  programmers  who  must  be  able  to  use  VESUB.  The 
VESUB  system  wiQ  be  used  by  Navy  instructors  who,  Uke  most  corrputer  users,  possess  a  basic 
understanding  of  computers,  but  are  not  computer  ejqjerts.  It  has  been  a  major  goal  during 
VESUB  development  and  formative  evaluations  to  design  the  most  usable  Instructor/Operator 
Station  (lOS)  possible.  The  next  sections  describe  the  results  of  formative  design  reviews  of  the 
VESUB  lOS. 

VESUB  lOS  DESIGN  REVIEWS 

The  VESUB  instructor  has  three  major  responsibilities:  1)  operating  the  system,  2) 
monitoring  the  trainee  during  training  exercises,  and  3)  creating  the  scenarios  to  be  used  in 
training  exercises.  The  first  two  responsibilities  must  be  accomplished  in  real  time,  while  the 
trainee  is  interacting  with  the  system  This  requires  the  instructor  to  watch  both  the  Instructor 
Station  screens  and  the  trainee.  If  the  instructor’s  attention  is  too  focused  on  operation  of  the 
system,  rather  than  monitoring  and  coaching  the  trainee,  it  is  likely  that  mistakes  will  be  made  to 
the  detriment  of  effective  training.  The  instructor’s  third  responsibility,  creation  of  scenarios,  is 
accomplished  off-line  with  the  system’s  Simulator  Generation  (SIMGEN)  software. 

Students  from  a  graduate  “Human- Con:5)uter  Interface  Usability  Evaluation  and  Design”  class 
at  the  University  of  Central  Florida  conducted  usability  analyses  of  the  VESUB  Instructor  Station 
dming  its  formative  evaluation  phase.  The  analyses  appfied  heuristic  usability  rules-of-thumb 
(Nielsen,  1993)  to  identify  usability  design  violations  in  the  Instructor  Station.  Before  discussing 
these  results,  the  next  section  provides  a  brief  description  of  the  VESUB  Instructor  Station. 

Description  of  the  VESUB  IQS 

The  VESUB  lOS  consists  of  three  computer  monitors,  each  with  their  own  keyboard  and 
mouse.  The  first  screen,  called  the  Simulation  Control  Display  (SCD)  is  shown  in  Figure  3  as  it 
appeared  at  the  time  of  the  usability  analyses.  From  the  SCD  screen,  the  instructor  controls  the 
environmental  configuration  and  other  features  of  the  simulation.  The  SCD  screen  also  provides 
the  capability  for  the  instructor  to  take  manual  control  of  the  trainee’s  submarine  or  to  allow  the 
trainee  to  control  the  submarine  via  the  voice  recognition  system  accessed  through  a  microphone 
located  on  the  bridge  mock-up.  Using  the  SCD  screen,  the  instructor  has  conplete  control  of  the 
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tme  of  day,  visibility  conditions,  sea  states,  wind,  currents,  and  other  visual  featoes  of  the 
simulation.  In  order  to  access  the  environmental  controls,  the  instructor  must  select  the 


Figure  3: 

VESUB  Simulation  Control  Display  (SCD)  Screen 


‘Environment  Control”  button  from  the  main  menu  in  the  upper  right  quadrant  of  the  screen. 

This  selection  brings  up  the  “Environment  Control”  window  in  the  lower  right  quadrant.  Figure  3 
shows  the  SCD  with  “Environment  Control”  selected.  Other  features  are  accessed  in  a  similar 
manner  when  they  are  selected  from  the  main  menu. 
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The  second  lOS  screen,  called  the  Visual  Simulation  Display  (VSD),  is  shown  in  Figure  4. 
Using  the  VSD  screen,  the  instructor  controls  the  placement  of  the  ownship  (the  trainee’s 
submarine)  and  all  the  traffic  ^ps  used  in  a  training  exercise.  From  the  VSD  screen,  the 


Figure  4: 

VESUB  Visual  Situation  Display  (VSD)  Screen 

instructor  has  access  to  aU  traffic  ships  and  can  monitor  any  preprogrammed  traffic  ships  or  take 
manual  control  of  any  ship  during  the  training  exercise.  The  left  side  of  the  VSD  screen  is  a  top- 
down  view  of  the  harbor.  This  view  shows  the  location  of  the  trainee’s  submarine  (ownship)  and 
all  of  the  traffic  sh^s  used  in  a  given  training  scenario.  It  also  diq)lays  geographic  features  of  the 
database,  such  as  the  channel  boundaries,  navigation  aids,  buoys,  piers,  etc.  Tbis  top-down  view 
can  be  adjusted  to  various  ranges  and  the  instructor  can  use  the  slide  bars  located  on  the  sides  of 
the  di^lay  to  pan  to  any  location  in  the  database  or  to  zoom  in  or  out  to  display  the  necessary 
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level  of  detail.  Although  not  shown  in  Figure  4,  the  controls  v^ch  allow  the  instructor  to  access 
and  control  the  traffic  ships  are  located  in  the  lower  right  quadrant  of  the  VSD.  Also  located  in 
this  quadrant  is  an  icon-based  interface  which  displays  specific  traffic  status  information  (e.g., 
angle  on  the  bow,  closest  point  of  approach,  range,  etc.).  In  the  upper  right  quadrant  are  similar 
controls  to  change  the  position  and  headmg  of  ownship  and  displays  with  information  on  ownship 
status. 

The  instructor  also  interacts  with  a  series  of  windows  located  on  the  monitor  of  the  main 
system  con:q)uter  (Onyx).  One  of  these  windows  is  used  to  turn  the  ^stem  on  at  the  beginning  of 
an  exercise  (see  Figure  5).  Then,  dtoring  the  exercise,  this  window  disappears  and  the  monitor 
displays  what  ffie  trainee  views  in  the  virtual  world  through  the  HMD  and  allows  the  instructor  to 
observe  the  critical  objects  that  the  trainee  is  watching.  After  a  scenario  has  been  conq)leted,  the 
screen  in  Figure  5  is  used  to  replay  the  exercise  to  provide  performance  feedback  to  the  trainee. 


Figure  5:  VESUB  Start-Up  Screen 


28 


Technical  Report  97-013 


USABILITY  DESIGN  HEURISTICS 

A  variety  of  guidelines  have  been  developed  to  assist  computer  system  designers  to  focus  on 
the  needs  of  the  user  (e.g..  Card,  Moran,  &  Newell,  1983;  Williges  &  Williges,  1984;  Hamel  & 
Clark,  1986;  Smith  &  Mosier,  1986;  Marshall,  Nelson,  &  Gardiner,  1987;  Helander,  1988; 
Shneiderman,  1992).  Similar  to  many  of  these,  Molich  and  Nielsen’s  (1990)  ten  usability  design 
heuristics  were  used  to  analyze  the  VESUB  Instructor  Station.  The  heuristics  are  as  follows: 

[1]  •  Simple  and  natural  dialogue.  Dialogues  should  not  contain  information  that 
is  irrelevant  or  rarely  needed.  Every  extra  unit  of  information  in  a  dialogue 
competes  with  the  relevant  units  of  information  and  diminishes  their  relative 
visibility.  AH  information  should  appear  in  a  natural  and  logical  order. 

[2]  •  Speak  the  users  ’  language.  The  dialogue  should  be  expressed  clearly  in 
words,  phrases,  and  concq)ts  familiar  to  the  user,  rather  than  in  system-oriented 
terms. 

[3]  •  Minimize  the  users  ’  memory  load:  The  user  should  not  have  to  remember 
information  from  one  part  of  the  dialogue  to  another,  histructions  for  use  of  the 
system  should  be  visible  or  easily  retrievable  whenever  appropriate. 

[4]  •  Consistency:  Users  should  not  have  to  wonder  whether  different  words, 
situations,  or  actions  mean  the  same  thing 

[5]  •  Feedback:  The  system  should  always  keep  users  informed  about  what  is 
going  on,  through  appropriate  feedback  within  reasonable  time. 

[6]  •  Clearly  marked  exits:  Users  often  choose  system  functions  by  mistake  and 
will  need  a  clearly  marked  ‘emergency  exit’  to  leave  the  unwanted  state  without 
having  to  go  through  an  extended  dialogue. 

[7] *  Shortcuts:  Accelerators — unseen  by  the  novice  user — may  often  speed  up  the 
interaction  for  the  expert  user  such  that  the  system  can  cater  to  both  inexperienced 
and  e?q)erienced  users. 

[8]  •  Good  error  messages:  They  should  be  expressed  in  plain  language  (no 
codes),  precisely  indicate  the  problem,  and  constructively  suggest  a  solution. 

[9]  •  Prevent  errors:  Even  better  than  good  error  messages  is  a  careful  design  that 
prevents  a  problem  from  occurring  in  the  first  place. 

[10] *  Help  and  documentation:  Even  though  it  is  better  if  the  system  can  be  used 
without  documentation,  it  may  be  necessary  to  provide  help  and  dociunentation. 

Any  such  mformation  should  be  easy  to  search,  be  focused  on  the  user’s  task,  list 
concrete  steps  to  be  carried  out,  and  not  be  too  large.  (Nielsen,  1993,  p.  20). 
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ANALYSIS  METHOD 

Fourteen  students  from  the  graduate  level  “Human-Computer  Interface  Usability  Evaluation 
and  Design”  class  at  the  University  of  Caitral  Florida  performed  the  usabihty  analysis  of  the 
VESUB  lOS.  The  class  was  divided  into  four  groups  and  each  group  conducted  an  independent 
analysis  applying  Nielsen’s  (1993)  usability  design  heuristics. 

ON-SITE  DATA  COLLECTION 

Data  for  the  analyses  were  collected  during  three  sessions  in  the  VESUB  laboratory  at  the 
NAWCTSD.  These  data  collection  sessions  allowed  the  student  analysts  to  iteratively  probe  the 
design  and  functionaUty  of  the  VESUB  lOS.  During  the  first  analysis  session,  the  VESUB 
research  team  provided  an  ioitial  orientation  demonstration  of  the  system,  followed  by  a  two-hour 
question  and  answer  period  that  focused  on  each  control  and  display  on  the  lOS.  The  analysts 
were  able,  in  an  unstructured  format,  to  ask  the  VESUB  research  team  as  many  questions  as 
necessary  to  fully  understand  each  lOS  function.  Because  of  the  complexity  of  the  VESUB  lOS, 
another,  three-hoirr,  question  and  answer  session  was  required  to  fully  cover  aU  of  the  lOS 
features  and  functionaUty.  A  third  session  with  only  the  analysis  group  leaders  was  conducted  to 
provide  more  detailed  information  on  various  lOS  functions.  During  this  session,  prints  of  the 
lOS  screens  were  provided  to  the  team  leaders  for  use  in  their  in-depth,  off- site  analyses.  The 
VESUB  research  team  explained  that  many  of  functions  and  features  of  the  lOS  could  not  be 
changed  in  the  technology  demonstration  system  due  to  schedule  and  budget  constraints. 
However,  the  students  were  told  that  the  results  of  their  analyses  would  be  reanalyzed  by  the 
research  team  and  used  as  recommendations  for  the  development  of  the  specifications  for  the 
follow-on  VESUB  operational  system 

OFF-SITE  ANALYSIS 

The  analysis  teams  took  the  data  collected  in  the  three  VESUB  laboratory  sessions  and 
collectively  assessed  the  VESUB  lOS  design  against  Nielsen’s  (1993)  design  heuristics.  The  first 
step  in  this  malysis  was  to  develop  VESUB  Instructor  Task  Flow  models.  These  task  flow 
models  and  the  VESUB  lOS  screen  prints  were  then  used  to  guide  the  more  detailed  usabihty 
analyses. 

Each  VESUB  lOS  control,  display,  and  instructor  task  was  evaluated  against  the  ten  design 
heuristics  and  those  which  violated  any  of  the  heuristics  were  classified  as  either  high,  medium,  or 
low.  Design  violations  classified  as  high  were  assessed  as  severely  hampering  system  usabihty 
and,  therefore,  should  be  resolved  prior  to  fielding  the  operational  VESUB  system  Medium  level 
violations  were  those  determined  to  require  redeagn,  but  were  judged  less  critical  than  the  high 
level  violations.  Design  problems  classified  at  the  low  level,  violated  one  or  more  heuristics,  but 
were  judged  as  not  to  inpede  the  practical  use  of  the  system  The  students,  compiled  then- 
analyses  into  four  reports  which  were  turned  in  for  class  credit.  The  reports  were  then  provided 
to  the  VESUB  research  team  to  use  as  deemed  appropriate. 
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POST-HOC  ANALYSIS 

During  the  question  and  answer  sessions,  the  VESUB  research  team  explained  to  the  analysts 
that  the  VESUB  system  was  still  under  development  and  that  many  of  the  lOS  usability  problems 
were  already  recognized  by  the  research  team  and  would  be  m^roved  in  later  iterations  of  the 
software.  Nevertheless,  the  students  conducted  their  analyses  on  the  lOS  as  it  was  configured 
at  the  time.  It  was  further  explained,  that  due  to  budget  and  schediile  constraints,  some  of  the 
lOS  design  violations  would  have  to  remain  in  the  demonstration  system,  but  would  be  corrected 
during  the  development  of  the  operational  VESUB  system 

Upon  receipt  of  the  analysis  reports,  each  member  of  the  VESUB  research  team  conducted 
an  independent  review  and  evaluation  of  the  identified  lOS  design  violations.  Based  on  their 
experience  with  the  development  and  use  of  VESUB  and  other  training  system  instructor  stations, 
the  team  members  selected  those  violations  that,  in  their  opinion,  were  most  detrimental  to  system 
usability  and  were  required  to  be  mq)roved  in  the  operational  VESUB  system  After  their 
s^arate  reviews,  the  research  team  met  for  focused  discussions  to  prioritize  the  lOS  design 
violations  and  select  those  to  be  highlighted  in  this  report. 
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RESULTS 

The  analysis  teams  produced  four  separate  analysis  reports  for  class  credit  and  also  provided 
copies  of  the  reports  to  the  VESUB  research  team.  Geary,  Cahill,  and  Bowen  (1997)  identified 
39  usability  design  violations,  22  of  which  were  designated  as  high  priority.  Biehl,  Pharmer, 
Rhodenizer,  and  Wohlschlaeger  (1997)  formd  21  design  deficiencies,  with  16  designated  as  high 
priority.  Lanham,  Rami,  Reeves,  and  Torizzo  ( 1997)  denoted  13  high  priority  items  from  the  29 
design  violations  they  identified.  Elaarag,  Tyler,  and  Vincenzi  (1997)  suggested  serious  redesign 
consideration  should  be  given  to  18  high  priority  items  from  the  32  design  violations  identified  in 
their  analysis. 

During  their  review  and  discussions  of  the  lOS  design  violations  reported  by  the  graduate 
students,  the  VESUB  research  team  prioritized  the  violations,  eliminating  those  that  were  either 
mm  nr  already  corrected,  or  that  were  based  on  the  student’s  incomplete  understanding  of  either 
the  system  or  the  training  task.  In  several  cases,  due  to  differences  in  interpretation  and  emphasis, 
the  analysts  identified  the  same  violation  under  different  heuristics.  The  VESUB  research  team 
regrouped  these  violations  rmder  the  most  relevant  heuristic.  The  results  of  this  post  hoc  analysis 
by  the  VESUB  research  team  are  shown  in  Table  2.  In  some  cases,  the  same  or  similar  violations 
were  identified  by  multiple  groups,  but  with  slightly  different  emphasis.  To  ensure  con^lete 
coverage,  these  similar  observations  are  shown  in  Table  2. 


Table  2 

VESUB  lOS  Usability  Design  Violations 


Analysis 

Team 

Usability  Heuristic 

1.  Simple  &  Natural  Dialogue 

2.  Speak  the  User’s  Language 

Lanham,  et  al. 

•  Information  not  presented  in  a 
logical  order 

•  Displays  irrelevant  or  extraneous 
information 

•  Not  all  the  information  is  expressed  in 
words  that  convey  the  appropriate 
message 

Elaarag,  et  al. 

•  'Information  Clutter”  (same  info, 
found  in  several  places) 

•  Interlinked  info,  on  each  screen 

•  Confusing  abbreviations 

Geary,  et  al. 

•  Extraneous  controls 

Biehl,  et  al. 

•  Redundant  information 

•  Confusing  labels 
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Table  2:  Continued 


Analysis 

Team 

Usability  Heuristic 

1 

1  3.  Minimize  the  User’s 

1  Memory  Load 

4.  Consistency 

Lanham,  et  al. 

•  DiJBBcult  to  Start  the  system  using 
multiple  keyboards 

•  Inconsistencies  in  color  and  shape  of 
controls 

Elaarag,  et  al. 

•  Long  start-up  procedure  with 
potential  for  errors 

•  No  prompts  for  right  mouse 
button  functions 

•  Different  scales  of  measurement  on 
each  screen 

•  Inconsistent  terminology 

Geary,  et  al. 

•  No  instructions  for  keystroke  controls 

•  Placement  of  controls  do  not 
follow  percentage  of  use 

•  Controls  activated  in  different  manners 

Biehl,  et  aL 

•  Too  many  displays 

•  Inconsistent  colors  when  controls  are  activated 

•  Not  consistent  with  “pull  down”  menu 
systems 

5.  Feedback 

6.  Clearly  Marked  Exits 

Lanham,  et  al. 

•  No  effective  feedback  for  user 

•  No  exits  to  back  out  from  options 

Elaarag,  et  al. 

•  Limited  info,  on  scenario  events 
during  run 

•  No  “Undo” 

Geary,  et  al. 

•  No  feedback  during  “freeze”  mode 

Biehl,  et  aL 

•  No  feedback  on  “clear  history^’ 

7.  Shortcuts 

8.  Good  Error  Messages 

DPDfRfKfKH 

•  No  shortcuts 

•  No  error  messages 

Elaarag,  et  al. 

•  Few  shortcuts  with  little  guidance 

•  No  warnings  for  “clear  history” 

Geary,  et  al. 

•  Autocenter  allows  own  ship  too  near 
screen  edge 

Biehl,  et  al. 

•  No  shortcut  for  adjustment  of 

HMD  eye  position 

9.  Prevent  Errors 

10.  Help  and  Documentation 

Lanham,  et  aL 

•  Few  preventative  measures 
against  errors  in  SIMGEN 

•  No  documentation  (at  time  of  analysis) 

Elaarag,  et  al. 

•  No  on-line  help 

Geary,  et  al. 

•  No  indication  that  right  or  middle 
mouse  button  functions  are  active 

Biehl,  et  al. 

•  No  ‘TJndo”  to  reverse  mistakes 

•  No  on-line  help 
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DISCUSSION  OF  lOS  DESIGN  VIOLATIONS 

Because  AME  developed  VESUB  using  the  lOS  screens  already  implemented  on  their  Virtual 
Ship  product,  and  because  of  budget  and  schedule  constraints,  many  of  the  usability  design 
violations  could  not  be  corrected  in  the  technology  demonstration  system.  Nevertheless,  the 
violations  discussed  below  diould  be  recognized  and  every  effort  should  be  made  to  reduce  or 
eliminate  as  many  of  the  usability  design  violations  as  possible  in  the  specification  for  the 
operational  VESUB  system. 

SIMPLE  AND  NATURAL  DIALOGUE 

Dialogues  should  not  contain  information  that  is  irrelevant  or  rarely  needed. 

Every  extra  unit  of  information  in  a  dialogue  competes  with  the  relevant  units 
of  information  and  diminishes  their  relative  visibility.  All  mformation  should 
appear  in  a  natural  and  logical  order  (Nielsen,  1993,  p.  20). 

Three  major  lOS  usability  design  violations  were  identified  under  this  heuristic:  1)  illogical 
organization  of  information,  2)  irrelevant  or  extraneous  information,  and  3)  interlinked 
information  on  multiple  screens.  Lanham,  et  aL  (1997)  pointed  out  that  the  information  on  the 
lOS  screens  was  not  ordered  in  a  “natural  or  logical  order  where  the  user  would  initially  gaze”  (p. 
5).  Persons  in  Enghsh  speaking  cultures  are  used  to  reading  from  left  to  right  and  top  to  bottom 
They  imconsciously  search  for  the  most  important  information  in  the  upper  left  comer  of  the 
screen.  The  controls  that  the  instractor  uses  most  often  (e.g..  Freeze,  Unfreeze,  and  Stop),  are 
placed  in  the  center  of  the  screen.  A  single  fix  for  this  problem  would  be  to  move  this  control 
box  to  the  upper  left. 

Three  out  of  the  four  analysis  groups  (Lanham,  et  al.,  1997;  Geary,  et  al.,  1997;  and  Biehl,  et 
al.,  1997)  stated  that  the  lOS  presented  irrelevant  or  extraneous  information.  An  exanple  of  this 
extraneous  information  is  found  in  the  displays  at  the  top  of  the  SCD  screen  shown  in  Figure  3. 
This  series  of  displays  shows  various  system  states.  These  same  states  are  also  found  on  system 
control  buttons  or  on  displays  in  other  areas  of  the  SCD  screen  or  on  the  VSD  screen.  For 
example,  the  run  mode  buttons  (Freeze,  etc.)  turn  white  and  change  labels  (e.g..  Freeze  changes 
to  Frozen)  when  activated.  This  information  is  also  shown  under  the  system  status  display  at  the 
top  of  the  screen.  Numerous  exatt^les  of  such  redundant  information  were  identified.  It  may  be 
possible  to  reduce  the  number  of  screens  needed  at  the  lOS  if  all  of  the  redundant  information  is 
eliminated. 

Elaarag  et  al.  (1997)  pointed  out  that  the  SCD  and  VSD  screens  display  inter-linked 
information.  This  can  cause  confusion  and  uncertainty  in  the  user,  who  should  only  have  to  look 
in  one  area  to  find  a  specific  piece  of  mformation.  This  is  especially  trae  in  early  versions  of  the 
software,  which  displayed  conflicting  information  (e.g.,  distances  in  feet  on  one  screen  and  meters 
on  another).  The  lOS  screens  should  display  universal  standards  of  measurement  for  the  ship 
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handling  task  (e.g.,  navigation  principles  require  use  of  latitude,  longitude,  yards,  nautical  miles, 
feet,  ^thorns,  etc.).  When  selected,  these  measures  should  be  consistent  across  all  screens. 

SPEAK  THE  USER’S  LANGUAGE 

The  dialogue  should  be  expressed  clearly  in  words,  phrases,  and  concepts  familiar  to  the  user, 
rather  than  in  system-oriented  terms”  (Nielsen,  1993,  p.  20). 

Lanham,  et  al.  (1997)  and  Elaarag,  et  al.  (1997)  found  problems  with  the  labeling  of  several 
controls.  Either  the  labels  were  con&sing  or  they  used  terminology  that  the  analysts  felt  would 
not  be  familiar  to  Navy  users.  For  example,  ih&Xmit  button  is  required  to  execute  a  variety 
of  selected  changes.  The  analysts  recommended  that  a  more  familiar  term,  such  as  “Execute,” 
replace  this  button.  Another  exarrq)le  is  the  two  controls  on  the  VSD  screen  (Figure  4)  that  are 
labeled  Vector  and  Project,  llese  controls  are  used  to  diq)lay  either  the  future  course  of  the  own 
sh^  based  on  its  current  heading  (Vector)  or  the  course  the  own  ship  is  ejq)ected  to  travel  over 
the  next  three  minutes  (Project).  The  analysts  recommended  that  these  buttons  be  relabeled  nging 
Navy  terminology  (e.g.,  “Current  Course”  and  “Proj  Course”). 

MINIMIZE  THE  USER’S  MEMORY  LOAD 

“The  user  should  not  have  to  remember  information  from  one  part  of  the  dialogue  to  another. 
Instructions  for  use  of  the  system  should  be  visible  or  easily  retrievable  whenever  appropriate” 
(Nielsen,  1993,  p.  20).  ^ 


Three  of  the  analysis  teams  (Lanham,  et  al.,  1977;  Elaarag,  et  al.,  1997;  and  Biehl,  et  al.  1997) 
determined  that  the  start-up  procedxire  was  too  long  and  complex  because  it  required  the  user  to 
interact  with  all  three  monitors,  keyboards,  and  mice.  These  con[q)lex  interactions  are  due,  in  part 
to  the  demonstration  nature  of  the  system  However,  the  VESUB  research  team  strongly  concurs 
with  this  analysis  and  will  recommend  that  the  operational  VESUB  systems  require  interaction 
with  only  one  keyboard  and  mouse  for  system  start-up. 

Elaarag,  et  al.  (1997)  and  Geary,  et  al.  (1997)  both  identified  a  deficiency  in  pronqits  for 
certain  controls.  In  some  cases,  the  right  or  middle  mouse  buttons  are  used  to  bring  up  displays, 
but  there  are  no  prompts  for  these  actions.  Keystroke  commands  on  one  of  the  keyboards  are 
required  to  initiate  some  actions  (e.g.,  turning  off’ fog  in  the  visual  scene).  There  are  no  pronqits 
for  these  keystroke  commands.  In  all  cases,  the  system  user  should  be  provided  with  an  easy  to 
understand  prompt  as  guidance  for  systems  functions.  These  prompts  could  be  automatically 

displayed  when  the  instructor  places  the  cursor  on  an  icon  or  the  instructor  could  point  and  click 
on  a  help  icon. 
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CONSISTENCY 

“Users  should  not  have  to  wonder  whether  dififerent  words,  situations,  or  actions  mean  the 
same  thing”  (Nielsen,  1993,  p.  20). 

It  is  essential  for  system  usability  that  its  interface  screens  be  designed  to  be  as  consistent  as 
possible.  The  analysts  found  several  inconsistencies  that  will  detract  from  system  effectiveness. 
Lanham,  et  al.  (1997)  and  Biehl,  et  al.  (1997)  foimd  that  some  controls  were  inconsistent  in  their 
color,  shape,  and  color  changes  when  activated.  Some  controls  change  from  blue  to  wiiite  vdien 
activated,  others  change  from  light  blue  to  dark  blue,  others  change  from  yellow  to  green,  and  still 
others  change  from  green  to  red.  These  inconsistencies  probably  stem  from  the  growth  and 
change  of  the  Virtual  Ship  system  during  several  years  of  development  by  various  system 
designers  and  programmers  at  AME.  Nevertheless,  all  controls  should  be  modified  to  display  the 
same  color  when  activated.  Furthermore,  the  controls  diould  be  modified  to  have  the  same  shape 
whenever  possible. 

More  problematical  than  the  color  inconsistencies,  Elaarag,  et  al.  (1997)  identified  different 
measurement  scales  on  each  screen.  This  is  another  artifact  of  the  development  of  the  Virtual 
Ship  system  by  a  variety  of  different  programmers.  Nevertheless,  improving  the  consistency 
across  screens  has  been  a  major  goal  of  the  VESUB  research  team  The  final  version  of  the 
VESUB  technology  demonstration  system  will  display  all  measurements  using  standard  Navy 
terminology.  This  will  also  be  true  for  the  operational  VESUB  systems. 

A  third  source  of  inconsistencies  was  identified  by  Geary,  et  al.  (1997).  They  found  that 
various  controls  were  activated  in  different  manners.  In  most  cases,  controls  are  on/off  toggles. 
However,  in  a  few  cases,  an  “off”  button  must  be  accessed  to  turn  off  a  feature.  The  specification 
for  the  operational  systems  will  include  a  requirement  for  aU  controls  to  be  conastently  activated 
and  deactivated. 

Each  group  of  analysts  indicated  that  usability  would  be  enhanced  if  all  controls  and  displays 
were  consistent  with  “pull  down”  menu  systems.  Since  many  users  are  familiar  with  standard 
“windows”  applications,  this  is  a  cogent  recommendation. 

FEEDBACK 

“The  system  should  always  keep  users  informed  about  what  is  going  on,  through  appropriate 
feedback  within  reasonable  time”  (Nielsen,  1993,  p.  20). 

All  analysts  identified  a  dearth  of  feedback  for  the  user.  This  is  especially  evident  during 
“freeze”  mode.  While  the  system  is  “frozen,”  the  instructor  can  change  various  system  states, 
such  as  visibility  or  time  of  day.  However,  no  actual  changes  occur  until  the  system  is  placed  in 
“run”  mode.  Furthermore,  no  indications  are  provided  to  remind  the  instructor  which  changes 
have  been  made.  Either  the  instructor  must  remember  a  potentially  large  list  of  changes,  or  make 
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them  one  or  few  at  a  time  by  constantly  switching  between  “freeze”  and  “run.”  Neither  option  is 
recommended.  The  operational  VESUB  systems  should  provide  immediate  feedback  on  any 
system  status  changes  made  by  the  instructor,  regardless  of  system  mode. 

Elaarag,  et  al.  (1997)  also  pointed  out  that  except  for  time,  no  feedback  on  events  in  the 
scenario  is  provided  to  the  instructor  during  a  run.  It  would  be  he^fiil  for  student  evaluations  if 
the  instructor  had  a  method  to  track  student  progress  and  to  determine  which  of  the 
preprogrammed  instructional  events  have  occurred  (e.g.,  closest  point  of  approach  (CPA)  of  each 
traflSc  ship,  degree  of  set  and  drift  of  ownship  due  to  currents,  bearing  of  each  traffic  ship  to 
ownship). 

A  potential  problem  with  the  “clear  history”  function  was  identified  by  Biehl,  et  al.  ( 1997). 
When  the  instructor  selects  Clear  History,  all  the  historical  tracks  of  own  ship  and  traffic  ships  are 
permanently  erased.  Some  form  or  warning  or  recovery  mechanism  should  be  available  for  this 
and  all  other  instructor  actions. 

CLEARLY  MARKED  EXITS 

Users  often  choose  system  fimctions  by  mistake  and  will  need  a  clearly  marked  ‘emergency 
exit’  to  leave  the  unwanted  state  without  having  to  go  through  an  extended  dialogue”  (Nielsen 
1993,  p.  20). 

This  heuristic  relates  to  the  item  discussed  above.  People,  even  instructors,  make  mistakes. 

The  current  configuration  of  the  Instructor  Station  provides  no  way  to  back  out  from  an 
unwanted  state  (Lanham,  et  al.,  1997;  Elaarag,  et  al.,  1997).  The  “undo”  function  and  other 
methods  to  correct  mistakes  should  be  included  in  the  specification  for  the  Instructor  Station  of 
the  operational  VESUB  systems. 

SHORTCUTS 

Accelerators  unseen  by  the  novice  user — may  often  speed  up  the  interaction  for  the  expert 
user  such  that  the  system  can  cater  to  both  inexperienced  and  experienced  users”  (Nielsen  1993 

p.  20). 

Once  a  system  user  becomes  an  expert,  he  or  she  may  want  methods  to  accelerate  various 
operations.  Lanham,  et  al.  (1997)  identified  no  operator  shortcuts  in  their  analysis.  Biehl,  et  al. 
(1997)  suggested  that  a  hi^  priority  shortcut  should  be  added  for  adjustment  of  the  eye  position 
in  ffie  HMD.  Currently,  the  instructor  must  use  the  keyboard  to  adjust  the  eye  position  for  each 
trainee.  A  setup  file  with  the  height  of  the  trainee  and  other  pertinent  information  (e.g.,  the  last 
scenario  the  trainee  successfully  completed)  could  be  provided.  With  such  a  file,  the  system 
would  automaticaUy  set  the  eye  position  each  time  a  given  trainee  logged  in  and  entered  the 
system 
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Elaarag,  et  al.  (1997)  identified  a  few  keystroke  shortcut  commands  that  are  currently  available 
in  the  system.  However,  there  are  no  instructions  or  indicators  for  these  keystroke  commands. 

As  with  all  system  fimctions,  clear  instructions  and  indications  of  system  states  diould  be  easily 
accessed  by  the  instructor. 

GOOD  ERROR  MESSAGES 

“They  should  be  expressed  in  plain  language  (no  codes),  precisely  indicate  the  problem,  and 
constructively  suggest  a  solution”  (Nielsen,  1993,  p.  20). 

Just  as  the  system  should  provide  exits  to  retreat  from  errors,  it  should  also  provide  warnings 
that  help  the  user  avoid  errors  or  identify  the  type  and  location  of  an  error  already  made. 

Warnings  are  often  designed  as  a  brightly  colored  or  blinking  window  that  asks  the  user  if  he  or 
she  really  wants  to  perform  the  selected  action.  This  gives  the  user  a  chance  to  reconsider  the 
action  and  the  possibility  to  change  his  or  her  mind. 

Lanham,  et  al.  (1997)  could  find  no  error  messages  available  in  the  system  Elaarag,  et  al. 
(1997)  pointed  out  that  the  lack  of  warnings  for  the  “clear  history”  fimction  is  an  especially  acute 
problem  As  discussed  above,  once  the  Clear  History  button  is  activated,  there  is  no  way  for  the 
instructor  to  recover.  There  should  be  a  clear  warning  for  the  instructor  after  the  Clear  History 
button  has  been  selected.  This  warning  should  require  at  least  one  additional  keystroke  before 
the  system  erases  the  history  information. 

Geary,  et  al.  (1997)  identified  a  problem  with  the  “autocenter”  fimction.  When  the  Autocenter 
button  is  activated,  this  fimction  automatically  centers  the  own  drip  icon  on  the  VSD  screen.  As 
currently  implemented,  the  own  ship  must  move  to  approximately  one  half  inch  from  the  edge  of 
the  screen  before  the  system  recenters  the  icon.  These  analysts  felt  that  autocenter  should  be 
activated  earlier  so  the  instructor  can  view  more  of  the  charmel  in  front  of  the  own  ship.  Perhaps 
the  “autocenter”  distance  should  be  selectable  by  the  instructor  since  different  harbors  or  chaimels 
within  a  harbor  may  present  unique  problems  that  require  different  default  distances  for 
“autocenter.” 

PREVENT  ERRORS 

“Even  better  than  good  error  messages  is  a  careful  design  that  prevents  a  problem  from 
occiirring  in  the  first  place”  (Nielsen,  1993,  p.  20). 

Lanham,  et  al.  (1997)  pointed  out  that  there  are  few  preventative  measures  against  errors  in 
the  scenario  generation  software,  called  Simulation  Generation  (SIMGEN).  Many  initial 
conditions  must  be  established  for  each  scenario  prior  to  accessing  the  graphic  editor  screen  to 
position  and  program  traffic  ships.  If  any  errors  are  made  during  the  definition  of  niitial 
conditions,  the  system  will  not  allow  the  instructor  to  move  to  the  graphic  editor.  Unfortunately, 
there  are  no  mechanisms  to  prevent  the  user  from  entering  erroneous  or  conflicting  set-up  data. 
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Furthermore,  when  an  error  has  been  made,  the  user  does  not  know  it  until  he  or  she  tries  to 
access  the  graphic  editor.  At  this  point,  the  only  way  to  find  the  error  is  to  go  through  the  text  of 
the  initial  conditions  file,  line  by  line.  This  is  extremely  time  consuming  and  fiustrating  for  the 
user.  The  operational  VESUB  systems  should  provide  internal  error  checking  mechanisms  that 
will  help  prevent  data  entry  errors  during  scenario  construction.  If  an  error  is  made,  the  system 
should  highlight  the  error  so  the  user  can  easily  correct  it. 

Geary,  et  al.  (1997)  pointed  out  that  the  user  interface  provided  no  indications  when  the  right 
or  middle  mouse  button  fimctions  were  activated.  Without  such  indications,  the  instructor  can 
forget  that  a  fimction  is  active  or  firrget  how  to  deactivate  it.  Unwanted  active  fimctions  promote 
screen  clutter,  which  can  precipitate  additional  errors. 

Biehl,  et  al.  (1997)  observed  that  the  inclusion  of  an  “undo”  fimction  would  help  prevent 
errors  by  affording  the  opportumty  for  the  user  to  reverse  mistakes  before  they  propagate  and 
cause  additional  errors.  As  in  previous  discussions,  the  VESUB  research  team  strongly  urges  that 
the  Instructor  Station  of  the  operational  VESUB  systems  include  an  “undo”  fimction. 

HELP  AND  DOCUMENTATION 

Even  though  it  is  better  if  the  system  can  be  used  without  documentation,  it  may 
be  necessary  to  provide  help  and  documentation.  Any  such  information  should  be 
easy  to  search,  be  focused  on  the  user’s  task,  list  concrete  steps  to  be  carried  out, 
and  not  be  too  large  (Nielsen,  1993,  p.  20) 

At  the  time  of  the  usability  analyses,  the  user’s  manual  for  the  VESUB  system  had  not  been 
con:q)leted.  The  VESUB  research  team  had  access  to  AME’s  Virtual  Ship  User’s  Manual  but 
this  document  did  not  iuclude  all  of  the  fimctionalrty  that  had  been  added  to  the  VESUB 
technology  demonstration  system  Lanham,  et  al.  (1997)  correctly  pointed  out  that  the  absence  of 
system  documentation  was  a  critical  omission.  Subsequent  to  the  completion  of  the  usability 
analyses,  several  iterations  of  the  VESUB  Software  User’s  Manual  were  produced.  The  VESUB 
research  team  used  inputs  from  Naval  Submarine  School  instructors  and  instructors  from  other 
submarine  training  facilities  to  help  ensure  that  the  user’s  manual  address  all  of  their  needs.  All 
shortfalls  in  the  technology  demonstration  user’s  manual  were  dociunented  and  used  in  the 
development  of  the  documentation  requirements  for  the  operational  VESUB  systems. 

Elaarag,  et  al.  (1997)  and  Biehl,  et  al.  (1997)  observed  that  the  VESUB  technology 
demonstration  system  provided  no  on-line  help.  On-line  help  can  be  a  very  efficient  mechanism  to 
answer  the  user’s  questions  about  a  specific  system  fimction.  Some  user’s  prefer  on-line  help  to 
consulting  a  user’s  manual  This  is  especially  trae  for  those  users  who  are  familiar  with  word 
processing  and  qiread  dieet  software  that  includes  this  fimction.  It  is  often  easier  to  keyword 
search  through  an  on-line  help  database  than  to  consult  the  index  or  table  of  contents  of  a  hard 
copy  user’s  manual.  In  some  on-line  help  systems,  the  user  only  needs  to  place  the  cursor  on  the 
area  where  he  or  she  needs  help  and  the  system  will  automatically  find  and  display  the  mformation 
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needed.  Of  course,  the  more  complex  the  on-line  help  system,  the  more  developmental  resomces 
win  be  required.  The  acquisition  team  for  the  operational  VESUB  systems  will  have  to  determine 
the  level  of  corcplexity  required  m  the  on-line  he^  system  Nevertheless,  on-line  help  is  a  feature 
that  is  recommended  for  the  operational  VESUB  systems,  regardless  of  the  level  of  complexity 
that  budgets  and  schedules  will  allow. 

SUMMARY  AND  RECOMMENDATIONS  FOR  THE  OPERATIONAL  VESUB 
SYSTEM 

The  previous  section  discussed  many  user-oriented  design  violations.  One  of  these  has  been 
corrected  in  the  demonstration  system,  others  have  only  been  partially  corrected.  Still  others  must 
be  corrected  when  the  operational  VESUB  system  is  designed  and  built.  To  facilitate  this 
process.  Table  3  summarizes  the  design  violations,  their  current  status,  and  the  implications  for 
training  if  they  are  not  corrected.  In  general,  the  10  S  should  be  designed  to  minimize  the  effort 
necessary  to  operate  the  system  so  the  instructor  can  devote  his  or  her  time  to  observing  and 
coaching  the  trainee.  All  of  the  recommendations  in  Table  3  lead  to  this  goal. 

Table  3 

Summary  and  Status  of  lOS  Design  Violations 


USER-ORIENTED  DESIGN 
VIOLATION 

STATUS* 

TRAINING  IMPLICATION 

1.  Simole  &  Natural  Dialoffue 

•  Illogical  organization  of  info 

•  PC 

•  Instructor  must  be  able  to  access  information  quickly 
so  attention  can  be  directed  toward  the  trainee.  The 
most  important  information  should  be  located  in  the 
upper  left  portion  of  the  screen,  least  important  in 
the  lower  right. 

•  Irrelevant  or  extraneous  info  or 
controls 

•  PC 

•  Only  necessary  info  should  be  displayed  so  instructor 
does  not  become  confused  when  looking  for  important 
information.  This  is  also  true  of  controls.  Only  one 
set  of  controls  should  be  required  for  each  function 
and  the  purpose  for  each  control  should  be  easy  to 
understand. 

•  Interlinked  info 

•  PC 

•  Instructor  should  only  have  to  look  in  one  spot  for  a 
piece  of  information 

MM  I  IfaUjUlUH  fcf 

•  Some  labels  use  unfamiliar 
terminology,  confusion  labels, 
or  abbreviations. 

•  PC 

•  All  controls  and  displays  should  use  terminology  that 
is  familiar  to  a  Navy  instructor  so  he  or  she  does  not 
have  to  guess  or  consult  the  user’s  manual  to 
determine  a  control’s  functionality. 

*  Note:  C  =  Corrected  in  demo  system;  PC  =  Partially  corrected;  OS  =  Requires  Correction  in  operational  system 
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Table  3:  Continued 


USER-OREENTED  DESIGN 
VIOLATION 

STATUS* 

TRAINING  IMPLICATION 

3.  Minimize  User’s  Memory  Load 

•  Complex  start-up  procedure 

•OS 

•  The  operational  VESUB  system  should  be  “turn  key,” 
it  should  not  require  complex  start-up  procedures. 

•  Lack  of  prompts  for  controls 

•  OS 

•  Instructor  should  receive  a  prompt  when  placing  the 
cursor  on  a  control. 

•  Placement  of  controls  should 
follow  percentage  of  use 

•  OS 

•  Instructors  should  be  able  to  most  easily  find  the 
controls  used  most  often.  The  most  important 
controls  should  be  located  in  the  upper  left  of  the 
screen,  the  least  important  in  the  lower  right. 

4.  Consistency 

•  Controls  do  not  have  same  colors 
for  same  functions  (e.g.,  on/ofi) 

•  OS 

•  Operational  system  controls  should  all  be  one  color 
for  a  given  state  (e.g.,  green  =  on,  red  =  ofO  to 
minimize  confijsion. 

•  Different  measurement  scales 

•  PC 

•  Measurement  scales  should  never  differ  on  different 
screens.  This  is  especially  important  when 
positioning  ships  during  scenario  construction  so  they 
appear  in  the  correct  location  during  the  exercise. 

•  Different  ways  to  activate 
controls 

•  OS 

•  Instructor  should  always  activate  controls  in  the  same 
manner  so  he/she  does  not  make  a  mistake  at  a 
critical  time  in  a  scenario. 

•  Controls  &  displays  should 
follow  "‘pull-down”  menu 
conventions. 

•  OS 

•  Most  users  are  familiar  with  ""pull-down”  menus,  like 
those  found  in  typical  word  processing  software. 

This  familiarity  will  make  it  easier  to  learn  and  use 
the  system. 

5.  Feedback 

•  Lack  of  feedback  especially 
during  system  “freeze”  state 

•  OS 

•  Instructor  should  not  have  to  wait  until  the  system  is 
in  ""run”  mode  to  see  whether  commands  have  been 
correctly  activated. 

•  Limited  real-time  feedback  on 
system  states  (except  time) 
during  a  scenario  run. 

•  OS 

•  Instructor  should  be  provided  with  trainee  and  system 
status  during  an  exercise.  Information  on  which 
critical  scenario  events  have  occurred  and  their  status 
is  required 

•  No  warning  on  “clear  history” 

•  OS 

•  Instructor  should  be  "Svamed”  any  time  he/she  tries 
to  take  make  a  nonrecover^le  action. 

6.  Clearly  Marked  Exits 

•  No  “undo”  functions 

•  OS 

•  Instractor  should  be  able  to  back  out  of  any  action. 

No  mistake  should  be  “fatal.” 

*  Note:  C  -  Corrected  in  demo  system;  PC  -  Partially  corrected;  OS  =  Requires  Correction  in  operational  system 
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Table  3:  Continued 


USER-ORIENTED  DESIGN 
VIOLATION 

STATUS* 

TRAINING  IMPLICATION 

7.  Shortcuts 

•  No  instruction  for  shortcut 
(keystroke)  commands 

•  OS 

•  Experienced  instructor  can  save  time  by  using 
shortcuts.  These  should  be  documented  in  the  User’s 
Manual. 

•  No  shortcut  for  HMD  and  other 
setup  procedures 

•  OS 

•  Instructor  should  be  ^le  to  quickly  and  easily  set  up 
the  HMD  eye  position  and  other  configurations  of  the 
system  for  each  trainee.  A  setup  file,  est^lished  the 
first  time  a  trainee  uses  the  system  could  reconfigure 
it  each  time  the  trainee  returns  for  additional  training. 

8.  Good  Error  Messages 

•  No  error  messages 

•  OS 

•  The  lOS  should  provide  error  messages  any  time  the 
instructor  tries  to  activate  a  function  that  will  cause  a 
major  change  in  the  system  state. 

9.  Prevent  Errors 

•  No  error  checks  in  “SEMGEN” 

•  OS 

•  During  the  construction  of  complex  scenarios,  many 
individual,  possibly  conflicting  items  of  information 
must  be  entered.  The  instructor  should  be  prompted 
when  an  error  has  been  made  and  the  system  should 
highlight  the  error  so  it  may  be  corrected  quickly. 

•  No  indication  when  right  and 
middle  mouse  buttons  are  active 

•  OS 

•  The  instructor  should  have  an  indication  when  any 
function  is  active.  Without  such  indications,  it  is  easy 
to  lose  track  and  make  mistakes. 

10.  Help  and  Documentation 

•  No  User’s  Manual 

•  c 

•  The  User’s  Manual  is  essential  for  training  of  the 
instructor  in  the  operation  of  the  system. 

•  No  on-line  help 

•  OS 

•  The  instructor’s  job  can  be  greatly  facilitated  with  the 
inclusion  of  on-line  help  functions.  He/she  should  be 
able  to  place  the  cursor  on  a  control  and  hit  help  to 
get  an  explanation  of  the  function.  This  will  allow 
inexperienced  instructors  to  continue  their  activities 
without  having  to  stop  and  look  things  up  in  the 

User’s  Manual. 

*  Note:  C  =  Corrected  in  demo  system;  PC  =  Partially  corrected;  OS  =  Requires  Correction  in  operational  system 
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RECOMMENDATIONS  FOR  FUTURE  RESEARCH  IN  lOS  DESIGN 

The  cortqjlexity  of  the  VESUB  technology  demonstration  system  and  the  necessity  to  build  on 
existing  products  resulted  in  niimerous  usability  design  violations  which  will  compromise  the 
training  ejffectiveness  of  the  operational  VESUB  system  if  they  are  not  corrected.  The  major 
focus  of  the  VESUB  demonstration  development  effort  has  been  on  the  functionality  of  the 
software.  This  was  necessary  to  meet  the  requirements  to  train  the  ship  handling  task.  Due  to 
schedule  and  budget  constraints,  less  effort  has  been  devoted  to  the  user-oriented  design  of  the 
lOS.  Many  of  the  design  violations  discussed  in  the  previous  section  could  have  been  avoided  if 
the  focus  during  system  development  had  been  directed  more  towards  the  needs  of  the  instructor. 
It  is  recommended  that  futme  computer-based  and  VR  training  system  developers  commit 
themselves  and  their  resources  to  inproved  system  usability  from  the  inception  of  system 
development.  This  level  of  commitment  will  help  ensure  consistency  and  usability  in  all 
components  of  their  instructor  stations. 

In  addition  to  the  VESUB  lOS  design  improvements  discussed  in  the  previous  section,  the 
following  are  recommended  areas  for  specific  instructor-oriented  research  and  technology 
development  which  will  increase  the  usability  of  future  training  systems.  This  list  is  not 
comprehensive,  but  includes  those  developments  that  tbe  VESUB  research  team,  based  on  the 
lessons  learned  during  ^stem  development  and  their  experience  with  other  training  systems, 
believes  to  have  the  highest  payoff  for  improvements  in  the  user-oriented  design  of  training 
system  instructor  stations. 

STANDARDIZATION  OF  lOS  GRAPHICAL  USER  INTERFACES  (GUIs) 

Most  instructors  today  are  familiar  with  basic  computer  functionality,  such  as  word  processing 
and  other  oflSce  productivity  software.  These  software  packages  have,  for  the  most  part, 
standardized  their  interfaces  on  the  “windows”  GUI  configuration.  Future  training  systems 
should  follow  this  lead  and  standardize  their  lOS  using  “windows”  style  GUIs.  These  interfaces 
should  be  built  aroimd  “pull  down”  menus  with  easy  to  tmderstand  labels.  These  interfeces  should 
follow  the  usability  design  recommendations  discussed  above  (e.g.,  be  consistent,  use  language 
the  instructor  understands,  include  emergency  exits). 

ON-LINE  HELP 

Another  software  feature  of  many  office  application  software  is  the  provision  of  an  on-line 
help  function.  On-line  help  allows  the  user  to  ask  questions  and  receive  answers  as  he  or  she 
interacts  with  the  system  This  is  more  efficient  than  having  to  stop  and  consult  written 
docmnentation  like  a  user’s  manual.  On-line  he^  does  not  eliminate  the  need  for  user’s  manuals. 
Rather  it  can  supplement  written  documentation  and  make  it  easier  for  the  instructor  to  remain 
focused  on  the  primary  task,  providing  instruction  to  the  trainee. 
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EMBEDDED  INSTRUCTOR  TRAINING 

In  recent  years,  greater  enq)hasis  has  been  placed  on  the  development  of  embedded  traioing. 
Embedded  Gaining  refers  to  training  that  is  included  in  or  added  on  to  operational  equipment. 

The  theory  is  that  embedded  training  is  highly  effective  because  it  is  delivered  in  the  same 
environment  and  context  where  operational  tasks  are  performed.  Thus,  transfer  from  training  to 
operational  performance  should  be  improved  by  effective  embedded  training. 

The  operational  tasks  required  of  instructors  is  running  a  training  system  and  providing 
training.  Any  conqruter-based  or  VR  training  system  should  include  embedded  instructor  training 
tools  that  provides  information  on  the  conoplete,  correct,  and  efficient  operation  of  all  training 
system  functions.  The  effectiveness  of  any  training  system  is  highly  dependent  on  the  capability  of 
the  instructor.  Embedded  instructor  training  should  greatly  unprove  the  instructor’s  ability  to 
optimize  the  effectiveness  of  the  training  equipment. 

TASK  ANALYSES  FOR  INSTRUCTOR  TASKS 

Task  analyses  have  traditionally  been  directed  toward  an  understanding  of  the  operational  task 
for  which  training  is  being  developed.  Enphasis  should  also  be  directed  toward  a  better 
understanding  of  the  instructor’s  tasks.  The  results  of  the  instructor  tasks  analysis  could  be 
provided  to  system  developers  early  in  the  design  of  the  training  system  so  the  necessary 
resources  and  technologies  can  be  brought  to  bear  on  the  requirements  of  the  instructor  and  the 
design  of  the  instructor  station. 

VOICE  RECOGNITION  AT  THE  lOS 

The  VESUB  system  has  successfully  demonstrated  that  speaker-independent  voice  recognition 
and  speech  synthesis  can  be  used  to  enhance  the  experience  of  the  trainee.  These  technologies 
also  have  the  potential  to  provide  missing  team  member  simulation  to  increase  the  flexibility  and 
effectiveness  of  team  framing.  Future  framing  systems  can  incorporate  voice  recognition  at  the 
mstructor  station  to  supplement  or  replace  many  point  and  click  actions.  For  example,  if  the 
instructor  observes  a  problem  during  a  scenario,  he  or  she  could  speak  the  word  “freeze”  to 
instantly  pause  the  scenario.  This  would  allow  the  instructor  to  move  away  from  the  lOS  to  more 
closely  observe  the  trainee  and  would  avoid  the  loss  of  time  that  would  occur  if  the  instructor  had 
to  grab  the  mouse,  locate  and  move  the  cursor  to  the  “freeze”  icon,  and  click  to  pause  the 
scenario.  Research  is  necessary  to  determine  which  instruction  activities  are  amenable  to  voice 
recogmtion  and  whether  inclusion  of  this  capability  will  in:q)rove  the  performance  of  the  mstructor 
and  the  effectiveness  of  the  whole  training  system. 
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CONCLUSIONS 

The  VESUB  technology  demonstration  system  has  successfully  integrated  VR  hardware, 
software,  and  coiurseware  to  provide  a  training  capability  for  the  submarine  OOD,  where  none 
previously  existed.  Developmental  constraints,  such  as  budget  limitations,  schedule  deadlines, 
and  technology  shortfalls,  have  bounded  the  fiinctionality  of  VESUB.  Nevertheless,  fleet  and 
school  ejq)erts  are  confident  that  VESUB,  based  on  their  interactions  with  the  system  during 
formative  evaluations,  will  be  a  viable  training  medixim  for  the  ship  handling  task.  It  is  expected 
that  the  results  of  the  traioing  effectiveness  evaluations,  which  will  be  documented  in  a  separate 
report,  will  validate  these  expert  opinions. 

Upon  the  coirpletion  of  the  VESUB  research  project,  the  next  step  is  to  provide  the  Navy 
with  an  effective  and  usable  operational  VESUB  system  The  operational  VESUB  system,  and 
other  future  VR  training  systems  must  achieve  a  balance  between  task  requirements,  the  latest 
hardware  and  software  capabilities,  proven  instructional  techniques,  and  user-oriented  lOS 
interface  design.  This  report  has  provided  information  to  help  build  better  instructor  stations  by 
presenting  the  results  of  usability  analyses  of  the  VESUB  technology  demonstration.  Similar 
design  violations  are  likely  to  occur  in  any  coroplex  simulation-based  training  system  They  can 
be  avoided,  only  if  the  needs  of  the  instructor  are  fectored  into  the  requirements  fijr  system 
development. 

The  lOS  design  violations  identified  iu  this  report  and  the  recommendations  for  their 
correction  will  be  used  to  improve  the  specification  for  the  lOS  of  the  operational  VESUB 
system  They  may  also  be  used  to  help  improve  the  design  of  the  lOS  in  other  fiiture  training 
systems.  Increasing  the  usability  of  the  lOS  to  allow  the  mstructor  to  more  easily  create,  monitor, 
and  control  traming  scenarios  will  significantly  contribute  to  the  training  effectiveness  of  VESUB 
and  many  other  training  systems. 
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COORDINATION 

The  VESUB  technology  demonstration  system  evolved  from  a  feasibility  demonstration 
system,  developed  xmder  the  Virtual  Environment  Training  Technology  project  at  NAWCTSD. 
This  feasibility  demonstration  was  used  dming  the  first  year  of  file  VESUB  project  to  solicit 
inputs  from  fleet  subject  matter  experts  (SMEs)  on  the  required  fimctionality  for  the  VESUB 
technology  demonstration  system  During  development  and  formative  evaluations  of  VESUB, 
numerous  fleet  SMEs  provided  guidance  to  ensure  that  the  system  was  realistic  and  provided  the 
necessary  fimctionality  to  support  training  for  the  ship  handling  task. 

The  development  of  VESUB  has  been  immeasurably  aided  by  the  support  and  guidance  of  the 
VESUB  Inqilementation  Planning  Group  (IPG).  The  IPG  included  members  from: 

•  NAVSUBSCOL  (N52),  Groton,  CT 

•  SUBTRAFAC  (Code  10),  Norfolk,  VA 

•  TRITRAFAC,  Kings  Bay,  GA 

•  SUBGRP  10,  Kings  Bay,  GA 

•  TRITRAFAC,  Bangor,  WA 

•  SUBPAC,  Pearl  Harbor,  HI 

•  SUBLANT  (N742),  Norfolk,  VA 

•  CNET  (T2223),  Pensacola,  FL 

•  CNO  (N879C),  Washington,  DC 

•  SUBTRAFAC,  San  Diego,  CA 

VESUB  has  been  demonstrated  to  hundreds  of  visitors  to  the  NAWCTSD  laboratories.  The 
success  of  the  integration  of  hardware  and  software  in  VESUB  has  led  to  the  use  of  several 
components  m  other  training  systems. 
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GLOSSARY  OF  TERMS,  ACRONYMS,  AND  ABBREVIATIONS 


AHD 

AME 

AOB 

AST 

CO 

COTS 

CPA 

GUI 

HCI 

HMD 

lALA 

lOS 

JO 

LADBM 

NAVAID 

NAWCTSD 

NTDS 

OOD 

SIMGEN 

SME 

SPAN 

VE 

VESUB 

VETT 
Virtual  Ship 

VR 

VSD 

XO 


Ahead 

Advanced  Marine  Enterprises  (Now  Nichols/Advanced  Marine),  VESUB 
System  Developers 
Angle  on  the  Bow 
Astern 

Commanding  Officer 
Commercial  Off  the  Shelf 
Closest  Point  of  Approach 
Graphical  User  Interface 
Human-Computer  Interfece 
Head-moxmted  Display 

International  Association  of  Li^thouse  Authorities 
Instructor/Operator  Station 
Junior  Officer 

Large  Area  Database  Management  System 
Navigation  Aid 

Naval  Air  Warfare  Center  Training  Systems  Division 
Naval  Tactical  Data  System 
Officer  of  the  Deck 

Simulation  Generation  Software  on  VESUB  and  Virtual  Ship 
Subject  Matter  Experts 

Submarine  Piloting  and  Navigation  Training  System 
Virtual  Environment 

Virtual  Environment  for  Submarine  OOD  Ship  Handling  and  Piloting  Trainiug 
System 

Virtual  Environment  Training  Technology  (6.2  Research  Program  at  NAWCTSD) 
AME’s  Commercial  Surface  Ship  Handling  Training  Product  (baseline  for 
VESUB) 

Virtual  Reality  (Synonymous  with  VE) 

Visual  Situation  Display 
Executive  Officer 
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APPENDIX  A 

VESUB  Hardware  and  Software 


HARDWARE 

Main  Computer  System:  Silicon  Graphics  Onyx  Deskade 

•  Four  R10,000  CPUs 

•  256  Mbytes  RAM 

•  One  Infinite  Reality  Gr^hics  Pipe  with  Two  Raster  Manager  Boards  (16  MB  Texture  Memory) 

•  Scene  Refi-esh  Rate  at  30  Hz 

•  Capable  of  Displaying  Approximately  21,000  Polygons  (fully  Z-buffered  and  anti-aliased) _ 

Instructor/Operator  Station  (lOS): 

•  Two  Silicon  Graphics  INDY  Desktop  Computers _ 

Head  Mounted  Display  (HMD):  n- Vision  Datavisor  HiRes 

•  Resolution:  1280  x  1024  pixels 

•  Field  of  View:  40  degrees  horizontal  and  30  degrees  vertical  (c^>able  of  stereo-optics  for  up  to  70  degrees 

horizontal  view) _ 

Head  Tracker:  Polhemus  3  space  Fastrack  (Magnetic) _ 

Printer:  Color  Postscript,  Manufacturer  To  Be  Determined _ 

Sound  System: 

•  Two  Radio  Shack  SSM60  Stereo  Sound  Mixers 

•  Two  Rane  MSI  Microphone  Amplifiers 

•  Two  Radio  Shack  Dynamic  CB  Microphones,  P/N  21-1172 

•  Two  Radio  Shack  Speakers,  Cat.  No.:  40-1324 


SOFTWARE 


Visual  Scene: 

•  Models  and  Tenain  Created  Using  ModelGen2  from  Multigen,  Inc. 

•  Real-time,  Interactive  3-D  Scene  Generation  Controlled  by  SGI’s  IRIS  Performer 

•  Marine  Visual  Effects  Created  Using  Vega  Marine  from  Paradigm  Simulation,  Inc. _ 

Instructor/Operator  Station  (lOS)  Interface: 

•  lOS  Screens  Created  Using  Visual  Applications  Builder  (VAPS)  from  Virtual  Prototypes,  Inc. 

•  Windows  in  SIMGEN  and  Start-iq)  Screens  Created  with  X-Designer  Release  4.5  from  Imperial 

Software  Technology  Limited  and  Data  Views  Corporation _ 

Audio  Effects: 

•.  Audioworks  from  Paradigm  Simulation,  Inc.  _ 
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APPENDIX  B 

VESUB  Training  Objectives 

PERCEPTUAL 

COMPONENTS 

TRAINING  OBJECTIVES 

IP.  Lcx^tingand 
Identifying 

Navigation  Aids 

•  The  trainee  shall  be  able  to  locate  navigation  aids  when  referenced  by  the 
navigator. 

•  The  trainee  shall  be  able  to  recognize  navigation  aids  in  the  visual  field 

2P.  Judging  Distance 

•  The  trainee  shall  be  able  to  accurately  judge  distances  to:  navaids,  contacts,  and 
landmarks. 

•  The  trainee  shall  be  able  to  judge  distances  relative  to  track. 

•  The  trainee  shall  be  able  to  verify  known  distances. 

•  The  trainee  shall  be  able  to  maintain  the  ship  within  the  acceptable  limits  of  the 
chaimel. 

3P.  Identifying  Start  and 
Completion  of  Turns 

•  The  trainee  shall  be  able  to  determine  relative  and  true  directions  on  a  compass. 

•  The  trainee  shall  be  able  to  determine  relative  bearings  to  the  navigational  aid  to  be 
used  for  turn  bearings. 

•  The  trainee  shall  check  turn  bearings  when  nimrTig 

•  The  trainee  shall  be  able  to  recognize  when  the  ship  is  clear  to  turn 
(e.g.,  when  buoys  are  in  line). 

4P.  Locating, 

Identifying,  and 
Avoiding  Obstacles 

•  1  he  trainee  shall  look  far  enough  ahead  to  evaluate  contacts  early. 

•  The  trainee  shall  be  able  to  recognize  new  contacts  prior  to  being  informed  of  the 
contact. 

•  The  trainee  shall  be  able  to  recognize  relative  directions  and  motions. 

•  The  trainee  shall  be  able  to  locate,  identify,  classify  and  differentiate  between 
various  types  of  contacts  and  other  obstacles. 

•  The  trainee  shall  take  early  and  effective  actions  to  avoid  obstacles  or  lessen  their 
negative  outcomes. 

5P.  Sense  of  Ship’s 
Responsiveness 

•  The  tramee  shall  understand  the  ship’s  capabilities  and  limitations,  including: 
advance  and  transfer,  speed  at  various  engine  orders,  loss  of  stearage  speed, 
distance  to  stop  or  reverse  course. 

6P.  Recognizing 
Environmental 
Conditions 

•  The  trainee  shall  be  able  to  accurately  estimate:  sea  state,  cloud  cover,  direction 
and  velocity  of  current,  wind  direction  and  speed,  and  time  of  day. 

•  The  trainee  shall  be  able  to  accurately  judge  the  state  of  visibility 

7P.  Recognizing 

Equipment  Failures 

•  1  he  trainee  shall  stay  alert  for  equipment  feilures. 

•  The  trainee  shall  regularly  monitor  rudders,  indicators,  and  other  equipment 

8P.  Detecting  &  Filtering 
Communications 

•  1  he  trainee  shall  be  able  to  recogmze  communication  sources  and  proper/improper 
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COGNTTIVE 

COMPONENTS 

TRAINING  OBJECTIVES 

1C.  Understanding  the 
Relationship  of 
Visual  Cues  to 
Chart(s) 

•  The  trainee  shall  be  familiar  with  all  the  navigation  aids  to  be  used. 

•  The  trainee  shall  understand  how  to  read  ranges  (fore  and  aft). 

•  The  trainee  shall  understand  how  to  determine  if  the  ship  is  left/right  of  track 
versus  left/right  of  range. 

•  The  trainee  shall  understand  when  to  attempt  to  drive  the  ship  in  the  center  of  the 
channel. 

•  The  trainee  shall  understand  buoyage  systems  (e.g.,  lALA  ‘‘A”  and  ‘"B”  systems). 

•  The  trainee  shall  understand  the  inaccuracy  of  buoys. 

•  The  trainee  shall  understand  the  accuracy/inaccuracy  of  Fix  information. 

2C.  Understanding 
Relative  Size  and 
Height/  Range 
Relationships,  and 
Angle  on  the  Bow 

•  The  trainee  shall  understand  how  to  determine  contact  mast  head  height. 

•  The  trainee  shall  know  his  height  of  eye. 

•  The  trainee  shall  know  how  to  determine:  size  and  distance  relationships  to  navaids, 
contact  length,  distance  to  the  horizon,  hull  down,  and  angle  on  the  bow. 

3C.  Understanding 
Advance  and 

Transfer 

•  The  trainee  shall  understand  the  concepts  of  advance  and  transfer. 

•  The  trainee  shall  understand  ship  characteristics  like  tactical  diameter  of  own  ship. 

•  The  trainee  shall  understand  the  criticality  of  turning  the  vessel  the  wrong  way. 

•  The  trainee  shall  understand  the  principles  of  conning  the  ship  through  turns. 

•  The  trainee  shall  understand  when  to  turn  the  ship  based  on  the  use  of  a  slide  bar. 

•  The  trainee  shall  xmderstand  the  principles  of  compensation. 

•  The  trainee  shall  compensate  for  set  and  drift  when  making  turns. 

•  The  trainee  shall  check  that  the  next  channel  is  clear  prior  to  turning. 

•  The  trainee  shall  not  drive  based  solely  on  the  Navigator’s  recommendations. 

4C.  Understanding  the 
Ejffects  of  Tides, 
Currents,  Wind,  and 
Seas 

•  The  trainee  shall  understand  how  the  wind  affects  the  height  of  seas. 

•  The  trainee  shall  understand  that  current  and  tides  tend  in  the  direction  of  the 
natural  geography. 

•  The  trainee  shall  understand  the  relationship  of  the  estimated  winds  associated  with 
various  sea  heights. 

•  The  trainee  shall  understand  that  sea  height  influenced  by  wind  speed  can  give 
false  indications  of  the  actual  direction  of  currents. 

5C.  Understanding  Rules 
of  the  Road 

•  The  trainee  shall  comprehend  the  criticality  of  Rules  of  the  Road. 

•  The  trainee  shall  correctly  exercise  Rules  of  the  Road  by  taking  appropriate 
actions  in:  overtaking,  meeting,  passing,  and  crossing  situations. 

•  The  trainee  shall  understand  the  rules  for  sound  signals  and  responses. 

•  The  trainee  shall  take  appropriate  action  when  nearing  a  bend  in  the  channel. 

•  The  trainee  shall  take  appropriate  actions  to  avoid  collisions. 

6C.  Understanding 

Relative  Motion 
(Direction  &  Speed) 

•  The  trainee  shall  imderstand  true  and  relative  bearing  and  their  significance. 

•  The  trainee  shall  be  able  to  convert  relative  to  true  and  true  to  relative. 

•  The  trainee  shall  be  able  to  determine  the  relative  direction  of  contacts. 

•  The  trainee  shall  be  able  to  determine  own  ship’s  motion  relative  to  fixed  objects. 
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COGNITIVE 

COMPONENTS 

TRAINING  OBJECTIVES 

7C.  Understanding 

Methods  to 

Differentiate  and 
Prioritize  Traffic 
Contacts 

•  The  trainee  shall  be  able  to  classify,  differentiate,  and  prioritize  various  types  of 
contacts  and  other  obstacles. 

•  The  trainee  shall  understand  safe  distances  to  hazards. 

•  The  trainee  shall  be  able  to  effectively  determine  contacts  of  interest. 

•  The  trainee  shall  correctly  assign  master  control  numbers  to  contacts  of  concern. 

•  The  trainee  shall  maintain  awareness  of  contacts  in  relation  to  own  ship. 

•  The  trainee  shall  prompt  personnel  for  supporting  information. 

•  The  trainee  shall  drop  contacts  of  interest  when  no  longer  of  concern. 

•  The  trainee  shall  be  able  to  correctly  determine  contact’s  angle  on  the  bow 

8C.  Understanding  Ship’s 
Operation  Under 

Harbor  Directives 

•  The  trainee  shall  understand  harbor,  port  limitations,  restrictions,  &  regulations. 

9C.  Understanding 

Methods  to  Deal  with 
Uncooperative  Traffic 

•  The  trainee  shall  take  proper  and  effective  actions  to  avoid  encounters  with 
uncooperative  trafBc. 

IOC.  Understanding 

Correct  Operation  of 
Ship’s  Systems 

•  The  trainee  shall  understand  the  correct  operation  of  bridge  equipment. 

•  The  trainee  shall  verify  rudder  orders  by;  visually  checking  the  rudder  and  the 
bridge  suitcase  indicator. 

•  Does  the  trainee  verify  engine  orders  by:  checking  the  bridge  suitcase  indicator 
and  observing  screw  wash? 

lie.  Understanding  When 
and  How  to  Take 
Corrective  Actions 

•  1  he  trainee  shall  understand  emergency  operating  procedures. 

12C.  Understanding 

Effective 

Communication 

Procedures 

•  The  trainee  shall  speak  clearly. 

•  The  trainee  shall  use  correct  terminology. 

•  The  trainee  shall  effectively  communicate  with  each  station  nging  required 
terminology. 

•  The  trainee  shall  acknowledge  all  reports  and  repeat  backs. 

•  The  trainee  shall  inform  appropriate  personnel  about  his  actions. 

•  The  trainee  shall  not  clutter  the  circuits. 
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